environment engineering is the new rubric engineering

@teortaxesTex @PrimeIntellect @georgejrjrjr not all verifiable but decent starting point

@teortaxesTex my interpretation of the technique in this paper is that you basically want more momentum and/or higher effective batch size a gradient only holds so many bits, increasing the # examples it needs to learn about focuses those bits towards generalization

@boazbaraktcs @jeremyphoward the exact opposite is true -- inference-time compute is provably sufficient to solve all problems solvable by any circuit, with steps scaling linearly in circuit size, given constant depth + log(size) embedding dim

@signalgaining the point isn't about using models to do rote calculations, it's about a more general paradigm of learning to solve increasingly hard problems without needing tons of solution data

@wordgrammer it would be pretty unusual if the CEO of a trillion dollar company was 20 years old

@Joanvelja that's a good point, if 10x-ing (?) RL lets you 0.1x test-time compute while still improving accuracy in general, that'd be awesome

RT @leonardtang_: i've been entirely consumed these past few weeks by the LLM-as-a-judge research agenda. there's lots of great work, but…

@Joanvelja they're still sampling 1K solutions for IOI and submitting 50 how good is o3 if you sample + submit 1 solution?

@StateSpeed_AB @casper_hansen_ different algorithms that do different things. GRPO is training on many distinct rollouts per prompt + computing relative advantages, sample count isn’t apples-apples for “training tokens”

@DavidFSWD oh just the latter haha (single node)

@TheXeophon @iScienceLuvr code execution

@TheXeophon i can only dream

the “wow that’s crazy” benchmarket is impossible to saturate until progress truly plateaus. it’s where you get a model to do something that makes people go “wow that’s crazy”

@TheXeophon they're a lot of fun to read sometimes

@jchencxh @rogerw0108 vLLM as in @vllm_project not Vision-Language Models

@rogerw0108 yes!