Kanjun 🐙🏡
@kanjun
Followers
17,379
Following
494
Media
88
Statuses
2,592
understanding human & machine minds to build a creative abundant future. CEO @imbue_ai . support founders @outsetcap . co-organize .
The Neighborhood (SF, CA)
Joined June 2009
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#KawalPutusanMK
• 2275705 Tweets
#TolakPolitikDinasti
• 1668132 Tweets
#TolakPilkadaAkal2an
• 1666372 Tweets
Nobara
• 237328 Tweets
FB JOIN OPENING BTV HOUSE
• 177648 Tweets
#BeyondTheVinesXFreenBecky
• 172415 Tweets
Mulyono
• 152122 Tweets
BATALKAN BUKAN TUNDA
• 147632 Tweets
Nigerians
• 104834 Tweets
Bolt
• 60412 Tweets
BOTO PARA SA SB19
• 53303 Tweets
鯉登少尉
• 36565 Tweets
チャイルドシート
• 33251 Tweets
ポケ森サ終
• 30400 Tweets
ポケットキャンプ
• 28194 Tweets
PilihDAMAI BarengPRABOWO
• 27911 Tweets
BBFA LAST HURRAH
• 24659 Tweets
LebihSEJUK LebihNYAMAN
• 23039 Tweets
首都高バトル
• 22923 Tweets
Widjiatno Notomihardjo
• 21605 Tweets
買い切り
• 19362 Tweets
中川大志
• 17270 Tweets
本人確認
• 13684 Tweets
GCSE
• 12896 Tweets
BTV X FORCEBOOK
• 11052 Tweets
月見バーガー
• 10617 Tweets
Pinned Tweet
We just released an extensive toolkit for training a 70B model from scratch, including 11 cleaned public reasoning benchmarks, a new code benchmark, 450K human labels, infra scripts for robust training on 2000 GPUs, and an optimizer for scaling small experiments to a 70B run!
Early this year, we trained a 70B model optimized for reasoning and coding. This model roughly matches LLAMA 3 70B despite being trained on 7x less data.
Today, we’re releasing a toolkit to help others do the same, including:
• 11 sanitized and extended NLP reasoning
6
82
458
1
5
47
It’s currently 2am, and I’m reading in a bookstore that’s also a bar. Why aren’t all cities like this?
324
1K
17K
I'm excited to share our $200M Series B at a $1B+ valuation to develop AI systems that reason!
We believe reasoning is the main blocker to effective AI agents. We train large models tailor-made for reasoning on our ~10K GPU cluster. On top, we prototype agents we use every day.
180
218
2K
Teaching is perhaps 100x harder than we expect. The common misconception is that it’s about sharing your knowledge; it’s not. Rather, it’s about understanding what’s in the student’s mind, modeling that in your mind, building a bridge from the student’s mental state to yours.
25
480
2K
I like people who try very hard. I find it odd that even as an adult, being "cool" seems to involve not caring too much and not appearing to try too hard, yet all the interesting things in human history have been done by people who cared a lot and tried unusually hard.
27
187
1K
Today, AI systems can create stunning art & beat humans at chess & Go. But they can't do things a 3-year-old can do. Why?
We're launching
@genintelligent
& open sourcing our research environment, Avalon, so as a community we can answer these questions:
53
186
965
In 2020,
@michael_nielsen
& I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer:
30
186
914
In the past year, I’ve talked to many bright, curious, ambitious people who believed they couldn’t do research. That was me, not long ago.
Here I share my improved understanding in hopes of offering a more empowering model of what research entails:
50
121
777
To guarantee an interesting conversation with any person, ask: “What is something you feel like you’ve thought about more deeply than most other people?”
13
89
768
Perhaps the job of a teacher should not be to convey information, but rather to kindle curiosity.
22
112
712
Reflections on Miami so far👇
1/ This city makes me feel shameless. I can wear anything, dance on the street, wear nothing, shout at the top of my lungs, and nobody will blink an eye. This is liberating!
15
32
480
For those who are asking - this is Backroom Bar, in Palermo Soho in Buenos Aires.
8
14
434
Spaced repetition gives you exponential returns for a linear increase in effort. It's hard to overstate exactly how powerful this is. One of the very, very few things in life that isn't subject to diminishing returns.
-
@michael_nielsen
+
@andy_matuschak
's
14
49
390
It’s remarkable how “not useful” Shannon’s discovery mapping relays to Boolean logic must’ve seemed at the time. It must’ve felt like a toy—“how nice, relays can do math”—because it wasn’t clear yet why being able to do binary arithmetic should matter.
13
57
388
“I want to have a positive impact” often means “I want to matter” — a selfish motive, masked as selflessness.
36
18
308
I’ve switched to entirely using “dedication” vs “discipline”, and it’s made a surprising difference to how I feel. Discipline feels hard, coercive, as if an outside force is making you act. Dedication, on the contrary, feels like a yearning, a pull from inside, rooted in desire.
10
32
285
Reflections on UK AI Safety Summit👇
1/ People agree way more than expected. National ministers, AI lab leaders, safety researchers all rallied on infrastructure hardening, continuous evals, global coordination.
Views were nuanced; Twitter is a disservice to complex discussion.
11
69
280
I suspect founding stories deter innovation. A coherent founding narrative can only be told after-the-fact, and is often simplified to sound linear. In the moment, when following instinct, I don't really know why I'm doing something, and the path is certainly not linear. 1/2
7
23
260
After DMT, I now suspect that many tribal rituals involving wailing, screaming, writhing, dancing were actually fantastic trauma release mechanisms that we’ve lost because they felt “primitive”. I wonder what it’d look like to bring these back into the cultural mainstream.
18
10
246
We've just announced a $150M deal with Dell to scale up model training!
Huge thanks to our partners at
@Dell
and
@VoltagePark
for getting this cluster up during a massive shortage of H100s 🦾
We’re hiring across product, eng, research, infra, & more:
We just announced a $150M deal with Dell to accelerate our work building AI agents that can reason!
3
8
76
7
23
235
Avalon is the world's fastest 3D simulator for RL agents. All baselines train on 1 GPU in ~1 day.
We want academic researchers to be able to study aspects of intelligence missing from today’s models, even w/o access to large-scale compute.
Get started:
2
42
199
It’s so hard to have both great execution and high curiosity at the same time!
10
14
173
There’s really something beautiful, magical, spacious about writing from midnight to 5am.
10
5
167
Does anyone know how people explained why cannons and guns had recoil before Newton's laws of motion?
I'm so curious about what kinds of (odd, I imagine) explanations were given! Like spontaneous generation and other explanations before Darwin.
21
14
163
4/ This city will be a hub for behavioral innovation, not technological innovation.
Fast-growing consumer apps & consumer packaged goods, not self-driving cars or robotics.
5
1
167
Communication is not about saying words; it's about taking responsibility for reconstructing the model in your head inside someone else's.
If the other person ends up with the wrong model, then you haven't communicated well!
4
28
165
@sashachapin
I can confirm :D beautiful cafes and restaurants etc open at all hours of day and night, very friendly people, and super cheap for us lucky US folks
2
1
147
Minsky: "We're in the thousand years between no technology and all technology. You can read what your contemporaries think, but you should remember they are ignorant savages."
3
17
152
Reading “classic” novels in high school seems a waste. I’d not yet had the life experience - or introspection - required to at all grasp what the authors were saying. Classics are deemed so because they capture the human experience; I hadn’t enough years to gather experience.
8
7
149
Research is simply a continuation of something we already naturally do: learning.
Learning happens when you understand something that someone else already understands.
Research happens when you understand something that nobody else understands yet.
1
20
150
Doing nothing is the foundation for doing something. A tense muscle cannot contract.
4
17
146
Given this, I’m surprised that teachers are expected to build their own curricula at all. It seems like it should take hundreds of hours to construct even a few weeks’ worth of material.
7
11
140
Good interface design for AI agents looks very different than for web. The important decisions aren’t about layout, icons, navigation; they’re about how to show work, communicate progress, understand preferences, edit plans, ask questions—much more like communication on teams.
6
16
139
3/ There is zero science. Every time I’ve discussed a technical subject in depth, it’s been with a Bay Area native.
I can’t imagine building a company that requires hard technical innovation here. The non-SF-native technical conversations have been inane.
5
0
138
If this resonates with you, we're hiring, both in SF and remotely!
And if you're a product engineer or a data engineer, we're particularly excited to chat right now :)
9
13
132
We're renaming to Imbue, to reflect our focus on imbuing computers with intelligence & human values. We aim to rekindle the dream of the *personal* computer—as truly intelligent tools that help us accomplish bigger goals in the real world.
Our approach:
1
7
128
Hypothesis:
@RoamResearch
is addictive because excellent search and bidirectional links resurfaces notes that surprise me enjoyably. This increases the perceived value of taking notes, causing me to take more notes. Brilliant,
@Conaw
:)
#roamcult
6
12
133
It’s often more important to understand the generator function for an idea, than the idea itself.
10
11
126
Not being "productive" may simply indicate that you aren't interested.
Instead of beating yourself up for low productivity, consider debugging why your work isn't interesting, or letting yourself do something more interesting.
3
17
125
"Undo" is a much better user experience than "Are you sure you want to <X>?" — noticed when comparing friction of Superhuman vs. Todoist.
4
4
121
If New York says “you should be richer”, Cambridge says “you should be smarter”, and Silicon Valley says “you should be more powerful”, which city says “you should be understanding new things that humanity does not yet know”?
Wondered while reading
@paulg
on cities & ambition
21
5
110
I've been collecting triggers for powerful creative contexts (since
@michael_nielsen
's essay: ). Surprisingly, *helping people* is a very reliable trigger: someone asks for advice, I start answering, and then *boom*, 1000 words & a bunch of ideas flow out.
5
0
108
Potential founders - especially young people - thus believe they need a good narrative ahead of time, when in fact what they need is to follow their curiosity and interest. Sticking to a narrative can actually be harmful - it ignores new data, and becomes less authentic. 2/2
3
4
105
Discipline is most sustainable & enjoyable when it stems from self-respect, rather than self-coercion.
7
7
108
Holy cow, Fourier and Lazare Carnot (thermodynamics Carnot’s father) were two of Napoleon’s generals! We think today’s distractions are bad…I can’t imagine trying to do mathematics while leading an army into battle.
4
7
107
6/ There’s a lot of name-dropping and arbitrary exclusivity. In SF, status is about whether you’re working on an ambitious project. In Miami, status is about whether you got invited to the hottest party.
2
1
102
When we see research as a process of understanding rather than a process of producing novel discoveries, it becomes clear that many curious, bright people who don’t believe they can be researchers already have the foundational skills needed.
2
10
99
Sharing my custom
@RoamResearch
CSS which I'm loving, with a wider right-hand panel and custom data-tags:
Built on
@Mappletons
's amazing work!
#roamcult
@Conaw
5
7
94
First principles thinking is simply about having an incredibly high bar for what counts as a “good enough” explanation, & tenaciously asking “why” until you get one.
If you let the explanation be good enough before you can visualize every piece, you‘ll stop asking why too soon.
3
11
96
I often hear people say that San Francisco is dying, and everyone is leaving, but my experience has been the opposite. Having visited various cities over the past year, I think SF is still, and perhaps increasingly, the only physical place in the world where I feel like I belong.
3
3
93
We're open-sourcing a PyTorch implementation of SIMONe—a recent approach towards unsupervised scene understanding from
@RishabhKabra
& others
@DeepMind
.
SIMONe creates remarkable segmentations from unlabeled video! For an explanation of the paper & code:
2
20
92
@tommycollison
I especially like Umberto Eco's take on this in "How to Travel with a Salmon". This chapter, "How to Justify a Private Library", is hilarious:
6
11
88
The process of science we’re taught in school - start with a question, generate hypotheses, test with experiments, draw conclusions - it’s technically correct, but it fails to capture the most important element: wanting to know the answer.
5
6
90
Avalon is a Featured Paper at NeurIPS!
We built Avalon so that academic researchers could make progress on general intelligence even without access to huge compute clusters.
Stop by our poster tomorrow 11am to learn more, or read on 👇
2
6
88
@michael_nielsen
We also wondered: why does nearly every new funding institution set out to "do things differently", and then, when observed 10 years later, seem to have snapped back to the same programs & decision processes as the NIH and NSF? What drives this pattern, and how do we break it?
1
6
78
One of my favorite passages of all time, from Feynman - thinking on this while noodling on sincerity, and what it means:
0
7
79
I think humility and confidence aren’t opposing traits. Humility is your level of openness to being wrong, and updating your beliefs. Confidence is when you predict high likelihood of success from your actions.
More humility -> better calibration to reality -> more confidence.
3
6
75
Put another way, the generating function for novel work is trying really hard to deeply understand something until you pass through the edge of current human understanding, and then continuing imaginatively onwards.
1
8
76
@RichardMCNgo
A few friends and I did a “doom circle” for a week where each evening, we’d pick one person and everyone else would tell them the biggest thing holding them back. I think this changed all of our lives, and I am a markedly different person as a result.
1
4
73
@michael_nielsen
An early question was: why do funders say they want high-risk, high-reward research, yet end up funding low-risk, run-of-the-mill work? Does high-reward research even need to be high-risk? Or could we construct processes to change the risk profile to enable low-risk, high-reward?
4
8
69
Typing extremely fast is a superpower worth acquiring. It increases the connection bandwidth between your brain and our current most powerful brain augmenter—your computer.
7
3
70
A solarpunk coworking/philosophy salon, communal library, cafe, and meditation studio with cozy reading nooks & wonderful company may be coming to a San Francisco near you ;)
If you'd like to come enjoy the space in July, please consider supporting it!
🚨 We’re excited to share that we’ve found a space for The Commons!
ft. a co-working space, philosophy salon space, communal library, cafe, meditation studio, and lots of cozy reading nooks.
We need your help in securing the space!
👉
🧵on our vision
11
40
282
6
6
67
If you’re interested in a principled approach to general intelligence, reach out!
Or come to our booth, presentation, or party at NeurIPS 🥳🎉
2
3
67
We suspect today’s AI chat interfaces are skeuomorphic, like how the first word processor programs had notepad lines.
We think interface invention can solve many core problems around agent robustness, trust, & collaboration.
4
2
63
"Beauty is everywhere, and cynicism is cowardly."
From Brian Moriarty's incredible introduction of Ray Bradbury:
0
5
63
Today’s AI systems are powerful, but they have limited ability to complete even simple goals for us.
Reasoning helps AI systems know when to ask questions, change approach, revise conclusions, break down problems, manage risk, & generally deal with the real world's complexity.
1
3
62
Probably my biggest learning hack is making Anki cards while watching lectures (not after!).
I'll write the question as the lecturer talks, and answer it on the spot. If I can’t answer it, I don’t understand, and I shouldn’t keep watching.
6
0
65
8/ You can work with a laptop anywhere. In SF, I get the evil eye for pulling my computer out in a cafe. In Miami, I’ve been working out of restaurants and hotel bars - people are very friendly and leave me alone. There’s less scarcity - there’s plenty of seating and space.
5
1
61
2/ SF sells efficiency, but Miami sells entertainment. The connection to consumer trends & what’s hot & what will sell feels reminiscent of LA.
By contrast, SF focuses on optimizing inefficiencies in consumers’ lives, which can be shortsighted.
5
1
59
When you try to understand something and hit the bounds of what humanity knows, suddenly, no one can tell you the answer.
If you care enough about the question, you have to answer it yourself; that's when you start running experiments and developing hypotheses.
1
2
61
5/ I encounter aspiring musicians everywhere: driving Ubers, lounging on the beach, chilling at bars. At least 5 musicians have walked up to me and struck up a conversation and played me their SoundCloud.
There’s a wonderful creative, “up and coming” energy.
1
1
61
9/ It’s a night owl’s paradise. The nights are beautiful, and encourage being outside & out late with others. The days are hot, and encourage sleeping in without feeling like you’ve missed anything.
2
0
59
Our ultimate goal: to engineer AI agents that can learn & understand like humans, so they can be safely deployed in the real world.
To do this, we'll extend Avalon over the next year to an encyclopedia of tasks that are easy for humans, but hard for machines.
5
3
57
We're lucky to be supported by backers who believe in long-term robustness & safety rather than short-term commercialization.
They've contributed over $20 million, and committed more than $100 million of future funding in a combination of options & technical milestones.
1
0
57
Why do humans need coping strategies, versus accepting and updating immediately? Where does the need to cope come from, and why is it adaptive? What’s the tradeoff - i.e. if you don’t need to cope and can just update, what are you losing?
18
5
57
The secret to staying curious is persistent dissatisfaction with your level of understanding.
This is harder than it sounds! We often remain dissatisfied in some areas of life (e.g. our work) but believe we understand in others (e.g. our relationships).
2
7
57
Self-acceptance is when you stop being the person you think you should be and start being the person you are.
5
4
56
We believe seriously using our agents every day is needed to push both reasoning models, and agent tooling & interface. Many of our agents write code, since that's our main work.
Ultimately, we aim to release systems that allow anyone to build robust, safe, custom AI agents.
1
3
53
It's much less important for kids to learn to write *well* than for kids to write *a lot*, without filter, as a form of self-expression. Once they write a lot, they'll inevitably learn to write well.
7
2
55
2/ Many were surprised by the subsequent summits announced in Korea & France. This essentially bootstraps AI dialogue between gov'ts—it’s brilliant.
With AI race dynamics mirroring nuclear, no global coordination = no positive future. This felt like a promising inflection point.
3
3
53
Context-switching makes life feel shorter.
(And by extension, so does attempting to multitask!)
5
4
52
4
5
51
It’s hard to do most things today—we've countless ideas & dreams that we never get around to. Computers help, but need micromanaging.
Our north star: truly personal computers that thin the barrier between idea & execution—giving us freedom, dignity, & agency to do what we love.
1
3
50
“the ultimate touchstone of friendship is not improvement. The ultimate touchstone is witness, the privilege of having been seen by someone, to have walked with them, believed in them, and sometimes, just to have accompanied them, on a journey impossible to accomplish alone”
2
3
50
Staying interested is simply about staying within the ideal band of novel information per unit time. Thus, reading and watching lectures can be never boring. If you’re bored, go faster or slower to maintain interest. 1/
7
10
51
So, ignoring your emotions actually means you're ignoring important information that your subconscious doesn't know how to put into words.
2
0
52
Introversion and extroversion are not two ends of a spectrum. They’re two somewhat independent spectra. One can be simultaneously very introverted and very extroverted. Not understanding this confused me for a long time.
3
1
50
Sometimes I seem to reach human interaction saturation, at which point each incremental person no longer feels interesting.
Hermit-mode solitude seems to resolve this, but I wonder at the mechanism that causes this in the first place.
5
2
48
I'm excited to share CARBS, our in-house hyperparameter optimizer, which has accelerated & automated our research on large language models and reinforcement learning.
CARBS automatically discovers scaling laws for every hyperparameter, plus huge performance & efficiency gains.
What if you could automatically discover both the scaling laws for every hyperparameter PLUS the best-performing network for every compute budget?
Introducing CARBS, a cost-aware hyperparameter tuner that does this & more: 👇
3
14
120
2
7
49
We assume that humans have bad intuition for exponentials, but I wonder if it’s just because we don’t get enough training data (given the natural world has few exponentials).
Perhaps a training sim for exponentials, given to kids, could result in much better intuition.
6
1
49
Reading good textbooks has made me realize how much of my life I’ve wasted reading bad textbooks 🤦🏻♀️
6
1
49
Had a very fun time discussing our collective AI future with
@reidhoffman
:) thanks
@Figma
for having me at
#Config2023
!
Recorded here:
1
3
47