I'm starting a "research in public" project on solving 20 Questions Games with LLMs and have some cool stuff to share! 🧵

Cool prompt idea: "Teach me something about <topic> that I don't already know. Something niche and obscure." But it turns out this is basically saying "hello llm, please hallucinate for me" Even when I asked it to cite a paper and include a quotation, it would make up a paper.

It's going to be so funny if we get AGI, but the only way to do it is test time compute scaling that makes it more expensive than humans

@Kylec1215 @TheXeophon It took 8 minutes

@Kylec1215 @TheXeophon Here you go!

@Kylec1215 @TheXeophon I like to use it for lit reviews and checking if projects are already done. Anything that requires searching through large amounts of information. Here's an example lit review I've also used it for health topics, e.g. research on the best way to exercise

@Kylec1215 @TheXeophon Sure!

@cccntu For comparison, curriculum learning (SFT with samples ordered by difficulty) also often leads to better generalization

@tensorqt Fun fact, the probability of that is 10^{-3.6x10^8}. It's so small you need exponents within the exponential notation. Anthropic really does have the mandate of heaven.

@iScienceLuvr From Karpathy himself: "It took me last ~6 weeks to get a from-scratch policy gradients implementation to work 50% of the time on a bunch of RL problems." source:

@OpenAI Feature request: In the sidebar, can we have a way to filter Deep Research chats vs. other chats? I want to return to Deep Research queries, but sometimes have trouble finding them in all the other chats

@nrehiew_ Just posted this in response to another rl post:

@nrehiew_ accuracy weighted too low so the output is just <answer> random int </answer> ?

@andersonbcdefg RL is the the automation of reward hacking

@TheXeophon @andrew_n_carr The key to AGI is giving the models imposter syndrome 😂

@I_loves_deep_nn Unhappy people send hate because they want other people to be on their level, and it's easier to make other people sad than make themselves happier. Fight back by emitting positivity!

@speedymachine1 Yep, I'm in the US, most of the ones here require a prescription

@IlyasHairline @abacaj This paper got test time scaling just by adding "wait" repeatedly

@abacaj Basically every LLM reasoning result looks so obvious in retrospect. Prompt engineering, CoT, STaR, Self-Consistency. Every paper your read is "why didn't I think of that"

@Aizkmusic I wanted to try and bought a pro subscription. It's really useful. lmk if you have a query you want to try