Jackson Stokes
@jackson_stokes
Followers
175
Following
353
Statuses
76
give me a break I’m on my journey
San Francisco, CA
Joined February 2012
@ross_cefalu @deepseek_ai Okay this is really pleasant to listen to and now I get the whole generated podcast thing
0
0
4
This is really interesting I think considering tokens per second, 24vs 8hrs work etc, a continuously running model could be ~100x as “productive” as a person Then effective batching means ~10 samples per clock cycle, so each gpu’s upper limit on productivity is like ~1000 knowledge workers? And 8bit quantization means 1b parameters=1gb ram, so models can be run on MacBooks etc Feels like the difficulty will still be making the slop generated by these models useful to us in some way
0
0
1
This is huge. Process Reward Models are the “brains” behind reasoning models like o1, and data annotation in PRM is very much unsolved. Excited to see more research be published in this space!
🚀 Exciting Advances in Process Reward Models (PRMs)! 🚀 Our latest research tackles the challenges of data annotation and evaluation in PRMs for better mathematical reasoning in LLMs. We show that MC estimation-based methods often fall short compared to LLM-as-a-judge and human annotations. 🔍 Key Findings: 1. MC estimation can lead to inaccurate step verification. 2. BoN evaluation strategies may inflate scores due to flawed processes. 3. Our consensus filtering mechanism integrates MC with LLM-as-a-judge, improving both performance and data efficiency. 📚 Blog: 💻 Hugging Face: 📊 ModelScope:
0
0
4
@paulg always prioritize runs over sets, wait to go out until there are other advantageous runs on the table you can build on, my dm’s are open if you need to phone a friend
1
0
5
Just let ChatGPT be Siri already. It wants to soooo bad
Today we’re rolling out a beta version of tasks—a new way to ask ChatGPT to do things for you at a future time. Whether it's one-time reminders or recurring actions, tell ChatGPT what you need and when, and it will automatically take care of it.
0
0
3