Eldar Kurtic
@_EldarKurtic
Followers
339
Following
746
Statuses
133
Compressing LLMs @NeuralMagic @RedHat & @ISTAustria
Joined July 2018
@TheXeophon @_philschmid @neuralmagic @AIatMeta Hey folks, you were interested in long-context evals when we released the quantization blog, so here’s a follow-up blog:
How well do quantized models handle long-context tasks? When we released the "Give Me BF16 or Give Me Death?" paper, the most common question we received was: What about long-context evaluations? To answer it, we adopted @NVIDIAAIDev’s RULER benchmark—and here’s what we found. 👇
1
0
2
@eliebakouch @DAlistarh @markurtz_ @neuralmagic Thanks for sharing our work! Two notes: 1. Not all quantization is naive. We optimized the process a bit (, but we didn't target long context evals in any way. 2. At 128K we do see a drop, but the BF16 is pretty bad as well which makes recovery very noisy
1
0
2
#EEML2025 coming to Sarajevo 🇧🇦 !
This summer, a group of elite AI scientists and engineers will come over to Sarajevo 🇧🇦 for #EEML2025 ❤️ We've been working hard behind the scenes to prepare a stellar program for our attendees! Trust me, you don't want to miss this 🚀 Applications open now! (details in post)
0
0
3
RT @neuralmagic: New blog: Discover how DeepSeek models achieve better performance and scalability with multi-head latent attention (MLA) a…
0
4
0
@MaziyarPanahi @huggingface If you would like to convert these raw scores into normalized ones (to be comparable to the official HF-Leaderboard), feel free to use this script: . Just run with: `python convert_to_leaderboardv2.py <json_output_from_lmeval>`
1
0
1
The full blog is available at: @neuralmagic @RedHat Quantized models with their quantization recipes for full reproduction are available at:
0
0
1
RT @RedHat: Today, Red Hat completed the acquisition of @NeuralMagic, a pioneer in software and algorithms that accelerate #GenAI inference…
0
35
0
The full paper (presented at EMNLP 2024) is available at: This is a joint work with @AmirMoeini99 and @DAlistarh, at @ISTAustria and @neuralmagic.
0
0
0
0
0
0
@srchvrs @neuralmagic Thanks for sharing our work! Yes, we have observed that accuracy on coding tends to improve a bit with quantiz. We don't yet have a clear explanation for why it happens, but it seems to be consistent. In addition to HumanEval blog results, we have observed this on MBPP as well.
0
0
2