Stephen Panaro
@flat
Followers
503
Following
1K
Statuses
819
making coffee and other things. @BrewTimerApp
boston
Joined May 2013
Despite the general messiness, it wrote one line of code in a different and simpler way than my solution. Neat.
0
0
0
@fleetwood___ Oh yeah. If this looks good for normal (scale/bias) quants, I am hoping it helps non-uniform/LUT methods too.
0
0
1
Possible silver lining: code could be a jumpstart if you want to try quantizing ModernBERT with SpinQuant.
0
0
0
@ivanchanavinah Dope! Text extraction is pretty good for the few I tried but formulas could use some work.
0
0
1
Tried a lot of prompt variations with o1-mini/preview so I do think it is just beyond them.
0
0
2
Might test this out. Seems low effort/fairly interesting. GPTQ, AWQ, llama.cpp, maybe HQQ. Anything others?
Seems like QuaRot/SpinQuant should be a prerequisite if you’re quantizing LLMs in any way. Having smaller weights/activations can only help. Right?
0
0
3
@CulStory @answerdotai Awesome. after the holidays I’ll take a look and see if I can get rotate working again.
0
0
0