msmarco hard negatives with Stella scores

@giffmana much better than jax imo, tensorboard profiling is terrible, much harder to track memory

@shaoruu @cursor_ai Scroll the chat without mouse 🙏

@qtnx_ on one hand we can’t expect them to truly keep up, on the other rip our classmates lmao you guys are cooked

@qtnx_ you got this, brother

@snowclipsed @moondreamai @vikhyatk beautiful, great work dog

@carlyfoxy a fun american tidbit that europeans find incredulous take the junk food receipt to school so they can afford more rulers

rewriting splax in torch - GPU profiling better in torch - don't need to rewrite models (like modernbert) - torch has sparse sparse matmul autodiff, jax only sparse-dense - heavyball optimizers will miss everything else though

torch has sparse sparse mm with backward but jax does not :(

@tct0620t Tarou, I am visiting Osaka in 2 weeks. I will be thinking of your posts on my walks. It will be my first time in Japan, so I am very excited.

@vikhyatk common failure mode with upwork engineer

sparsity is shape rotator final boss

just skip to the bottom lol

@qtnx_ another bitter lesson

@tekbog >we all know bro has an alt in tpot that'll throw em off

do IR people run NanoBEIR for validation during training runs? couldn't do it every thousand steps but wouldn't be THAT slow

@carlyfoxy wow based

@carlyfoxy Generalized learning punished (AI too good), extremely specialized learning rewarded (we can still just beat AI and other people will trust us more) also if we don’t learn we are soulless

@carlyfoxy no