Pavel Surmenok
@surmenok
Followers
2K
Following
75K
Media
91
Statuses
6K
Autoilot / AI at Tesla
Redwood City, CA
Joined July 2009
FSD V13: point-to-point self-driving without touching steering wheel or pedals. A large deep neural network trained with a large dataset end-to-end: photons in, controls out.
FSD 13 leaves parking lot (+ awkward interaction with other driver). the smoothness is absolutely INSANE. it also saw the Model 3 backing up before I did, I was wondering why it wasn’t moving lol
5
13
275
Once upon a time, I interviewed a seasoned ML engineer, asked him “what do you think about batch norm”. He looked at me with eyes full of painful memories and laughed. Then I knew that he is an expert.
7
2
152
@dividendology One is sum of all savings, another is growth rate. 2nd image as sum of savings would look more like this. Still noisy, but not as much.
2
5
143
@saurabh_shah2 The story is wild. @JeffDean just wanted to save bandwidth and chopped off the lowest 16 bits of fp32.
1
2
106
@cremieuxrecueil What surprised me: even men from Denmark have quite high proportion of 18%.
10
0
89
@theorizur Have you tried to work at startups, or at orgs that move fast (e.g. pretty much any Elon’s company)?.
7
0
80
@ID_AA_Carmack Not unlike Windows which had all kinds of patches for bugs in 3rd party apps. “On beta versions of Windows 95, SimCity wasn’t working in testing. Microsoft tracked down the bug and added specific code to Windows 95 that looks for SimCity. If it finds SimCity running, it runs the.
3
7
82
A large model trained on enough data learns sophisticated behaviors. The network just wants to learn, give it more compute and data.
FSD Supervised 13.2 reverses to exit parking spot blocked by delivery truck, then waits for oncoming traffic to clear before proceeding. This all happens implicitly within the model, which is trained on extensive data of similar real-world scenarios.
1
4
70
@mualphaxi @Stanford It might be worth to find authors of the posters and make a clear permanent searchable record of their actions.
4
0
66
Almost every startup at YC Demo Day is building with LLMs. Huge change comparing even with the previous demo day.
6
6
57
@finbarrtimbers Kaplan et al found that (for pretraining) the learning rate schedule is irrelevant as long as LR summed up over all training steps is large enough, includes a warmup period and decay to near-vanishing value at the end.
2
4
56
@stylewarning I’d start from writing tests. Then see if it’s well modularized or it’s a ball of spaghetti, attempt to refactor in the latter case.
1
0
49
@_apoorvnandan @karpathy Ok, actually step 1 is “become one with the data”. But verifying the loss of randomly initialized network is correct is the first thing to do before starting training.
1
1
50
In the Arena today. Trying stuff. Some will work, some won’t. Always learning.
5
0
46
Now general public will learn about mighty Q-learning algorithm.
reading in between the lines, is Q* the fabled breakthrough in AlphaStar-style search + LLM that so many big labs are trying to get working? Many research projects in GPT-4 self-verification + search have not yielded really strong performance improvements, so I'd be quite.
0
0
40
Saturday morning. Good time to check how my training jobs are doing. GPUs go brrrr.
4
0
39
0
4
36
@GarrisonLovely Reading the article, it looks more like Hoduras became a nightmare for Honduras and wants to ruin it by walking back the deal they previously agreed on. They deserve to be bankrupted if that’s the case.
2
0
32
@Carnage4Life @BeanstalkFarms So it’s not stolen then, the protocol worked as designed. Fascinating.
0
0
29
@GergelyOrosz @t3dotgg @ThePrimeagen I don’t joke about bus factor. I’m very serious about bus factor.
0
0
28
@Crypto_uWu @growing_daniel H1-B is a temporary worker visa, issued for 3 years, can be renewed for 3 more years. After 6 years they have to get out of the country (unless apply for a green card or some other visa type). They can’t bring family except a spouse and kids under 21yo.
2
1
27
I had a TODO to buy more NVDA. Seeing -16% this morning felt like a Christmas gift.
2
0
22
0
0
23
OpenAI board members cleaned up their social media profiles: Tasha McCauley closed off Twitter, Helen Toner and Adam D'Angelo don't mention OpenAI on LinkedIn.
1
3
23
@peterrhague Honestly I thought it’s your real photo, AI augmented. That’s odd that some people are mad about EVs. EVs are great.
8
0
22
@growing_daniel @Crypto_uWu They can’t. They cannot even immigrate in that visa, it’s a non-immigrant visa by definition.
@Crypto_uWu @growing_daniel H1-B is a temporary worker visa, issued for 3 years, can be renewed for 3 more years. After 6 years they have to get out of the country (unless apply for a green card or some other visa type). They can’t bring family except a spouse and kids under 21yo.
7
0
20
1
3
21
@dkrajendra Rare elements are not rare, that’s misnomer. They are everywhere in the Earth crust. Professing these metals is not environmentally friendly (much pollution), so we outsource it whenever possible.
0
0
21
Next gen Tesla Bot. The future is already here!.Great job @_milankovac_ and the team!.
There’s a new bot in town 🤖. Check this out (until the very end)!.
0
0
21
1080Ti is still economically better than Titan V if you run CNNs.
1
7
18
@karpathy Problem with comments is that they get out of sync with code. Best code is self-documented. Comments should not explain what the code is doing, but may explain why, e.g. reasons for unconventional usage of something something as workaround for a bug somewhere.
3
0
18
@twobitidiot @theallinpod @rabois Try @BG2Pod , people on the street say that it has vibes of early All-in pod. I enjoy it, information dense, no bullshit.
0
0
19
- What is Occam's razor?.- Well, the simplest explanation is that there is a guy named Occam and it is his razor.
3
0
19
@finbarrtimbers GPU utilization is a bad metric in practice. GPU utilization can be 100% while GPU does nothing but waiting for e.g. NCCL communication from other ranks. GPU power consumption is more informative.
3
0
17
I wish Google to publish a thorough postmortem to explain what went wrong with aligning their chatbot and how they are going to fix it. Curious how much of it are explicit instructions in the system prompt vs. RLHF.
3
0
18
@KareemRifai Like elections in Russia in 2011 when pro-Putin party won, and votes in one region (as displayed on TV) summed up to 146%.
1
0
18
Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better.
2
0
18
Eval of LLM systems is conceptually similar to eval of other machine learning models. Look at predictions (ideally on distribution of inputs from your real users), identify patterns of errors, cluster/categorize errors, develop evals for each error cluster.
I started doing office hours on LLM evals and met with 8+ founders in the last 3 weeks. Common questions:. - Which components of our app do we start evaluating (RAG,tool calls, etc)? .- What metrics should I use?.- Where should I spend my time? . All have the same solution.
0
1
18
Thank you @chazman .V13 is 🔥.
I don't do posts like this very often. just read it please. Since I have been home after my redeye flying all night from PHX . My @Cybertruck and Model Y had received Supervised FSD v13.2.1 while parked, over the air cellular (OTA) for free. I got in my Cybertruck dead tired.
1
1
18
A story about a black SFFD firefighter assaulting his Asian colleague. The department tried to cover it up, the victim was fired, the assaulter kept his job. So much dysfunction in SF public services.
Black privilege in SF: . Black firefighter looks up Asian coworker’s address, shows up at his house and tries to beat him to death with a wrench. Asian firefighter gets fired for cooperating with police. Black firefighter keeps his job, never missing a paycheck.
1
0
16
@nikitabier I’ve owned a house for less than two years, and it’s relatively new and recently renovated, but I already have phone numbers for good repairmen for all kinds of things.
0
0
15
More goodies from DeepSeek. Janus-Pro is a novel autoregressive framework that unifies multimodal understanding and generation.
deepseek just dropped some new models . people are still getting used to R1. Janus-Pro is a novel autoregressive framework that unifies multimodal understanding and generation. It addresses the limitations of previous approaches by decoupling visual encoding into separate
3
4
17
@YunTaTsai1 Trump’s willingness to go to the long form podcasts is respectable. You can feel what kind of person he is, what points are important for him. Listen for a couple hours and you can make a better informed decision whether to hire him.
1
0
17
@sirbayes Interesting. I never heard of the other meaning of inference. Prediction seems a bit off. Prediction is about the future. For example, you can predict where a pedestrian will be 1 second from now. But detecting where they are now is not prediction. I hesitate to use the word.
2
0
16
@srush_nlp We should normalize pseudonyms and links to arbitrary webpages. Democratizing science.
0
0
16
Sometimes the model answer is wrong. Sometimes the model answer is correct but we just don’t like the result.
2
0
16