Wrote a brief blog on RLHF based on my independent study into RL with @obastani -- implementing DPO from scratch, aligning GPT-2, comparing to SFT. Takeaways and code in 🧵

For a 100 page PDF, gpt-4o on ChatGPT is about twice as fast as 2.0 Flash on Gemini. Weird because Gemini’s latency is usually way better. My guess: Gemini is doing OCR on a screenshot of each page, but OpenAI parses text out. This is definitely still an open problem

@anishtxt Agree, building AI for consultants -> building a AI native consultancy

Whats the best way for driven students to get hosting / GPU / API credits these days :D

@anishtxt Who knew PDF would be the biggest headwind for useful AI

Snapchat is clearly hanging on by a thread given their last ditch monetization efforts Notifications of stories from influencers i dont follow, sending me direct snaps from brands…this is sloppy

@karpathy It's not clear to me why DeepSeek's RL process is so dependent on CoTs. Couldn't the same recipe (i.e. GRPO on verifiable rewards) help the model output better code out-of-the-box? Without a CoT?

@sahiladhawade True in the long run

@aniruthn Just where I grew up and saw a ton of smart kids go into CS

Has anyone studied what day to send emails to maximize p(reply) for diff demographics This would be sooo valuable

@DaveShapi AI will help here

Came across this cool app that turns images of empty rooms into furnished rooms using diffusion. Any vision folks have guesses as to the approach? Idk what models can do such seamless, constrained generation.

@rohindhar Is it fair to say that post-liquidity event some people will move further south to start families in a more suburban setting? Ie would demand increase more in SF or the peninsula

@saurabh_shah2 Surely leetcode adds these kinds of questions soon 🙏

@denny_zhou LLM generation can be thought of as a bandit which makes comparison a bit easier

@morgancheatham How do you see this playing out with insurance? Personally don’t know how many people can afford cash for imaging, but if they could, would insurance start approving recommendations from health apps?

Code available at Blog also available on substack: