@jon_barron @polynoamial @OpenAI i think your comment gave me much clarity on Noam's meta-point which I kinda missed the first time I read his tweet.

you were/are the Chief Scientist of Meta, and a FAIR Lead -- where both Zetta and Llama were located; I think characterizing any team within your direct influence in a bad light in public is not nice. yea the Llama folks were great. praise them. What if Zetta was allowed to run like Llama (left alone, without leadership changing their mind weekly, put under undue constraints), maybe the counterfactual would be different -- lets leave it to speculation. A lot of the Zetta folks went on to do great work.

how legit is the research? more plausible that it's hooking onto something pathological but irrelevant, like say "female faces are on average a different shape than male faces, so the angle of capture (or some other silly, but irrelevant difference) of retinal images is slightly different because of how they fit in the imager's harness.". like the army tank classifier folklore.

RT @Thom_Wolf: Finally took time to go over Dario's essay on DeepSeek and export control and to be honest it was quite painful to read. And…

RT @allen_ai: Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of t…

@jxmnop unclear tbh, data mix is not specified. probably is the big reason for performance

if you find david beating goliath to be a bizzare surprising story, read history? if you think state of the art in AI cannot possibly be achieved without massive resourcing, go read the AlexNet paper or read about Alec Radford's accomplishments?

deepseek is rewriting history rn!!

@_arohan_ pinged you on workchat!

@AnushElangovan @__tinygrad__ fwiw, if @realGeorgeHotz was offering to work on my stuff for free, and wanted two boxes; I'd drive them out to him myself. @__tinygrad__ is a beautiful, phenomenal piece of software IMO.

@heyjchu fwiw that spend plan sounds incredibly sketch, and if they manage to raise that'd be impressively bubbly, and the investor needs to look themselves in the mirror.

RT @andrew_n_carr: want to train your own model with a million token context window? the great torchtitan team has implemented pass-KV ri…

@abhijit_MLab India has some of the most well-capitalized startups; so i didn't talk about money. AI base models are being built by well-capitalized companies across the world, not academic research labs. hence i feel money isn't that relevant of a dimension here.