Core Francisco Park
@corefpark
Followers
585
Following
263
Statuses
81
Physics of Intelligence @ Harvard Physics. Currently working on: Agents
Cambridge, MA, USA
Joined May 2023
New paper! “In-Context Learning of Representations” What happens to an LLM’s internal representations in the large context limit? We find that LLMs form “in-context representations” to match the structure of the task given in context! 1/n
20
180
1K
Right now there are so many interesting questions to answer about fundamental principles of intelligence itself! If you are a researcher who is deeply interested in unraveling the governing principles of intelligence, join us!
There's never been a more exciting time to explore the science of intelligence! 🧠 What can ideas and approaches from science tell us about how AI works? What might superhuman AI reveal about human cognition? Join us for an internship at Harvard to explore together! 1/
0
0
4
RT @lateinteraction: More Qwen. I'm increasingly comfortable saying these papers seem to be a discovery of some sort about Qwen models, not…
0
30
0
The role of RL is probably to teach the model what backtrackings are good backtrackings, i.e. when to backtrack and how to backtrack. 3/n
0
0
4
A few thoughts: 1) The base model should backtrack at least a bit. 2) The pretraining data must have backtracking for 1) to happen. 3) 1) will be probably stronger if the base model was trained with distilled data. (Which I think is the case for Qwen) 2/n
0
0
0
This!!!! Qwen 7B base can already do backtracking!
@DimitrisPapail @lateinteraction We’ve thrown all algorithms we have at this problem, including PPO and MCTS, over the last 3 years. All of them saturated. What changed is what goes in the “base” model. Literally thousands of papers on this, idk how its a discussion.
0
0
3
@jiayi_pirate Thanks for open sourcing this!!!! This is an enourmous contribution of understanding the science of RLxLLMs!!
0
0
1
RT @jiayi_pirate: We reproduced DeepSeek R1-Zero in the CountDown game, and it just works Through RL, the 3B base LM develops self-verifi…
0
1K
0
@ma_nanye Hi! I'm wondering if this naturally solves the 6 finger problem (and similar problems of this nature)!
0
0
2
Wondering if this solves fingers... (without training the reward model explicitly for that :) )
Inference-time scaling for LLMs drastically improves the model's ability in many perspectives, but what about diffusion models? In our latest study—Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps—we reframe inference-time scaling as a search problem over sampling noises. Our results show that increasing search computation can further enhance generation performance, pushing the capabilities of diffusion models further. 🧵[1/n]
0
0
2
@atemyipod My guess is that there literally was a bug during the data upload? its not this just one row.
0
0
0
RT @Napoolar: 🎭Recent work shows that models’ inductive biases for 'simpler' features may lead to shortcut learning. What do 'simple' vs…
0
108
0
Absolutely amazing read! This paper is a must read if you are interested in reasoning and inference scaling!
We have a new position paper on "inference time compute" and what we have been working on in the last few months! We present some theory on why it is necessary, how does it work, why we need it and what does it mean for "super" intelligence.
0
0
5