"if you can compress data well, you understand its patterns" Low loss on training error + highly compressible hypothesis = strong generalization Large models tend to find compressible hypotheses in their weight space because of the optimization process (gradient descent) and because the training distribution can actually be compressed well. (Language is highly regular, i.e. a low length hypothesis/model exists.)

RT @omarkilani: YOLO Monday I guess... 🤫

@ozenhati @fire @GroqInc Actively looking at various options 🤞

RT @GavinSherry: Huge news here in Saudi Arabia — amazing partnership between @GroqInc and Saudi Arabia 🇸🇦

@TrelisResearch Yeah if RL on LLM base is learning how to reason for LLMs I hope their reasoning gets better over time too not just that it learns how to reason for longer (albeit coherently/productively – which to be fair also isn't trivial).

RT @mbalunovic: We finally have an answer to the debate over whether LLMs generalize to new math problems or they merely memorized the answ…

@noamgat Jevons paradox, more value per token increases the addressable market 🙌

Read the entire thing, trust me, you won't regret it.

Ah, did find: >Software Engineering Tasks: Due to the long evaluation times, which impact the efficiency of the RL process, large-scale RL has not been applied extensively in software engineering tasks. As a result, DeepSeek-R1 has not demonstrated a huge improvement over DeepSeek-V3 on software engineering benchmarks. Future versions will address this by implementing rejection sampling on software engineering data or incorporating asynchronous evaluations during the RL process to improve efficiency.

And the crucial bit of what is "emerging on its own" in the long CoT RL paradigm: >backtracking, checking, error-correction and other branching-like behavior

@0xs____ Will pass on this feedback re: rate limit handling. Ty

@TrelisResearch @anaisbetts >things probably collapse into being one agent I feel like encapsulation and information/context overload contradict this. How to organize I don't know, but "putting everything into one thing" feels like it won't scale.

@AlexGDimakis @ryanmart3n

I like to think this is inspired by the human brain that is pretty sample efficient also. Evolution = pre-training. Eduction = supervised learning. Trial and error self teaching = RL.

77% woah higher than usual 🔥

RT @hila_chefer: VideoJAM is our new framework for improved motion generation from @AIatMeta We show that video generators struggle with m…

RT @ggerganov: @Olney1Ben @exolabs Or just use llama-rpc: