Brad Neuberg
@bradneuberg
Followers
11K
Following
167K
Media
1K
Statuses
28K
ML eng @planet. Mentor w @FDL_AI. Prev @ Dropbox/Google. Made coworking. Interests: ML, space, EO, VR. https://t.co/m7fXSRYQW3
San Francisco, CA
Joined May 2008
“Hope is like a road in the country; there was never a road, but when many people walk on it, the road comes into existence.” — Lu Xun.
12
34
261
Musk *wants* to get in a fight with the San Francisco Board of Supervisors, and they are going to walk right into this one. It won’t look good for them. There’s a lot of other more important things they should be spending their time on (housing, affordability, safety, reviving.
469
155
4K
“Today, in a paper published in Nature, we share the discovery of 2.2 million new crystals – equivalent to nearly 800 years’ worth of knowledge. We introduce Graph Networks for Materials Exploration (GNoME), our new deep learning tool that dramatically increases the speed and.
84
591
4K
Wow so evidently the quarter that Meta got slammed for expenses & earnings it turns out Zuck was actually massively buying available GPU compute on the market to be ready for scaling AI, increasing expenses. Got to appreciate Zucks conviction & willingness to look bad to the.
30
170
3K
It’s over for leet code software engineers.It’s just beginning for idea guy software engineers.
65
122
2K
Was Marvin Minsky single handedly responsible for slowing down the growth of neural networks by perhaps 20 years? Even though Minsky actually studied and wrote on neural nets he ultimately was one of the major evangelists for symbolic approaches, which mired the field of AI in a.
144
146
2K
Terence Tao, the famous mathematician, on using LLMs to aid in mathematical research:. "2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process. When integrated with tools such as.
23
268
2K
The Apple Vision Pro UI is skeumorphism of the iPhone and iPad user interfaces, to act as a training wheel bridge for people entering spatial computing, in the same way the original iPhone UI was skeuomorphic of physical objects. This generation is comfortable with 2D iPhone UIs,
46
145
2K
If you are a software engineer and aren’t paying a measly 20 dollars a month for ChatGPT Plus for assured access to GPT4 you obviously don’t value your time and productivity enough. 20 dollars is like 2.5 coffees.
234
83
1K
AI demos are easy, AI products are hard. Anyone can slap together an AI demo that seems to work on the surface & impresses a crowd, but true AI products have to have quality, generality, be fast, hit the right features, not have domain drift, etc. I'm not impressed by AI demos.
97
210
1K
There’s a certain subset of people that want to use climate change to completely change everything, ending 500 years of capitalism, fundamentally restructure society, etc. I think this is a distraction & dangerous: we need to laser focus on just solving climate change.
64
80
987
AMD (@AMD) has no idea how much value they are leaving on the table with this. The legendary @realGeorgeHotz was literally willing to fix their software to be competitive with NVIDIA for deep learning & it seems they are dropping the ball. AMD: if you can't write the software.
54
47
1K
I’ve voted Democrat in every election of my adult life, but I have to say the deep techno-pessimism and doomerism of the Democratic Party has given me pause on whether they will support advancements in energy, abundance, AI, space, and more. The disaster that is Democratic.
269
15
998
We had to invent AI in order to actually understand ffmpeg options.
44
63
980
I think this is one of the biggest submarine stories of our time, as it’s results and impact doesn’t line up neatly with existing tribal political affiliations. It’s somewhat of an Uncomfortable Result. Basically we seem to have been accidentally doing sulfur based geoengineering.
pretty crazy:.- container ships burn fuels that emit a lot of sulfur.- the sulfur seeds clouds, increasing the reflectivity of earth, cooling it.- new climate rules in 2020 limit sulfur emissions by cargo ships.- a lack of ship-clouds may explain anomalous heating this year.
33
129
978
If bicycles were just invented now and appeared on the streets of San Francisco the SF Board of Supervisors would outlaw them.
14
143
879
They built a 13 Watt (!) implementation of a billion parameter network running on an FPGA using this scheme.
This is really a 'WOW' paper. 🤯. Claims that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales and by utilizing an optimized kernel during inference, their model’s memory consumption can be reduced by more
16
81
866
@testaccountoki BTW this is how a Finnish person says “good work”. Their communication style is different.
6
11
840
@JaimeOrtega Yes I agree, if SF had a functioning government it would do all of those (the sign is obviously annoying). However SF does not have a functioning government, and will focus on the sign despite other important problems.
30
2
818
Assuming Google is now in its 10 year “lost in the wilderness” Balmer period, if you were CEO of Google & could make any change to help shift Google to its next, positive Satya Nadella phase what would you do?.
216
29
814
ChatGPT is incredibly powerful as a tutor. I used it this morning for it to teach me more about Kalman filters, had it explain the equations to me, had it cast it into Python & Numpy since I learn better from code.
21
64
746
Kevin Murphy’s new 2021 edition of Probabilistic Machine Learning e-textbook is out
3
162
644
This is surprising: “Research published recently suggests that Pfizer’s drug Viagra can decrease the risk of developing Alzheimer’s disease by up to 69 percent.”
64
40
618
The instability at OpenAI is yet another reason for open source AI - do you really want to have so much power concentrated in the hands of one or two organizations, where whoever is at the helm (or whoever plots behind their backs to seize power) wields so much control and power.
25
82
614
No joke - if we had not conflated nuclear power with nuclear weapons in the 1980s, & continued developing fission, *WE WOULD NOT CURRENTLY HAVE CLIMATE CHANGE*. That’s pretty wild actually, can you even imagine?.
35
49
607
@PicturesFoIder “Sorry, you have to have a security clearance to know what I did during that gap in my resume. Let’s just say I saved many lives.”.
3
2
542
@PaulSkallas Secretary of State tends to be part of the Cursus Honorum of appointments to eventually become consul, I mean President.
5
1
567
@Molson_Hart Y Combinator is predicated on startups that require low capitalization since most of the technology they depend on is already mostly developed. This was true pre-2015 where startups depended on commodity open source software, the smartphone capitalized by Apple and others, cloud.
26
33
548
Open source project named Quivr that indexes your local files on your machine & allows you to query them with large language models. I want something like this but directly integrated into my Macs Apple Notes + all my browser tabs & history, local on PC
16
72
522
I want a Unix command line utility that is a local LLM that I can point at a local directory tree (like a GitHub checkout for example or a program I’m working on), then I can ask that program questions about my code or help, everything done locally. Does this exist yet? It can.
69
25
488
This LLM guide from @brexHQ is actually very good, hat tip to @kevinweil for making me aware of it
5
80
463
This is some of the best science fiction I’ve heard for awhile, kind of like Harry Potter really (but it’s definitely not hard science fiction). But seriously, it’s amazing how many AI grifters want to use an AI doom and gloom narrative to try to leverage the power of the state.
How a US/China superintelligence arms race will play out:. “The CCP is going to have an all-out effort to infiltrate American AI labs. Thousands of people, the full force of the Ministry of State Security. There's an enormous incentive for a first strike.”. @leopoldasch
85
37
476
“We found 52,000 new layered compounds similar to graphene that have the potential to revolutionize electronics with the development of superconductors. Previously, about 1,000 such materials had been identified. We also found 528 potential lithium ion conductors, 25 times more.
4
29
467
This is convenient: example notebook from OpenAI showing how to take a subset of embeddings you care about, then learn a simple transformation matrix on top of these to force them to be closer to each for things like cosine similarity. It’s essentially a lightweight learned.
11
46
465
The United States should rename itself to “AAAUnited States” so it shows up first when choosing it in country drop-down pickers on web forms.
66
21
444
If you don’t think DallE-2 and Imagen are an Alexnet level moment in the machine learning world you aren’t paying attention enough. Very impressive visual results coming out of these. Getting similar chills to when I saw first web browser, iPhone, etc.
15
80
424
Could we make GPT-4 class LLMs much cheaper to train by allowing core weights to just do reasoning, & externalize storing facts into external database + using tools? I believe GPT-4 is overparameterized & essentially storing entire compressed Internet, seems like waste of space.
48
32
419
Reading through the Segment Anywhere Model (SAM) paper, some notes & insights as I do so in this Twitter thread. This whole paper is quite incredible actually, very strong work 1/n .
5
59
416
@RyanKhurana @skooookum It’s all about intrinsic motivation: if you love learning about the world do it, if not don’t. Don’t wait or expect external rewards for “being a good student”. Me, I love learning, constantly, it’s one of my hobbies, I don’t care if world cares.
1
16
349
Terraforming Mars could be easier than scientists thought. Tiny rods made from martian ingredients could trap heat and warm planet. "Injecting tiny particles into Mars’s atmosphere could warm the planet by more than 10°C in a matter of months, researchers find—enough to sustain.
26
41
378
Blue Origin should abandon their rocket program, throw themselves completely into using SpaceX’s Starship program, & completely dedicate themselves to O’Neilian space manufacturing & colonies, truly delivering on the vision of orbital factories & living.
58
27
357
@Arnhem77 What about all that open air drug dealing and fentanyl literally happening a block down the street over at Civic Center? Maybe they can do something about that too?.
109
3
347
I’ve always found Monte Carlo Tree Search interesting, it’s common place in many of DeepMinds papers. Always surprised I don’t see it showing up in other places. Here’s a nice beginners guide to it.
9
40
357
What's something you know and believe, a contrarian take to what everyone else believes right now, that you know 10 years from now everyone will share & agree with you, and that it will actually just be conventional boring wisdom then?.
295
25
342
@mikeyhemlok @MarkHarrisNYC @chrisgeidner The sequel to Oppenheimer will be called “Teller: Say Hello to My Little H-Bomb!”.
2
7
327
Come on @elonmusk, every one of these likes is obviously from a thirst-bot. It's really messing up the community on X, you've got to do something about this.
72
12
329
This is a really cool paper: differentiable kernel sizes!. Being used in this work to perhaps replace Transformers with convolutions:
9
56
334
0
8
315
Of any of the large computer companies (Google, Amazon, Meta, Apple, etc) the last true believer in open source is Mark Zuckerberg:.- PyTorch.- React.- Llama.Etc. Prove me wrong.
34
18
324
So what's the story with @midjourney only being accessible through a Discord server? Seems like a really strange tech choice IMHO, just put it behind a web API of some kind?.
45
7
315
“Rapidly developing new technologies based on these crystals will depend on the ability to manufacture them. In a paper led by our collaborators at Berkeley Lab, researchers showed a robotic lab could rapidly make new materials with automated synthesis techniques. Using materials.
5
15
318
@pastasnack_e Total wealth and productivity from Roman Empire to modern age, beginning to climb roughly around first Industrial Revolution?.
6
1
312
From Google antitrust suit brought today: "Google falsely told publishers that adopting AMP would enhance load times, but Google employees knew that AMP only improves the [redacted] and AMP pages can actually [redacted] [redacted] [redacted].".
8
81
297
It looks like President Carter didn't cancel the Space Shuttle development program in the 1970s because the Space Shuttle was key to monitoring arms control agreements with the Soviet Union, excellent historical reporting from @SciGuySpace:
12
44
293
I believe these only get like 30 minutes or so of battery life. What does the technology path of 8+ hours of energy life on these kinds of drones look like? What kinds of energy technology breakthroughs are needed for that?.
81
8
291
@moultano I believe Newton’s early calculus notation was very heavy to work with, we work with a descendent of Leibniz’s notation which was superior.
1
2
282
“TorchGeo is a PyTorch domain library, similar to torchvision, that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data.”
5
46
284
At the root of physical reality is a single nested, complex numpy expression 500 characters long. Good luck understanding & unpacking it though.
18
14
283
@nikitabier “Virtue signaling”. I’m not saying the idea doesn’t have merit but people saying it usually drip with smug.
22
3
242
If you think the only way to solve climate change is to have everyone become vegetarians living in tree houses conserving energy in the fall and spring, it ain’t gonna happen and we are going to blow right through the 2050 warming targets.
14
21
261
Back in the day (pre-SpaceX) many New Space companies would fail to get investment & die because potential space company investors would call their uncle “who works at NASA” & ask their opinion, & NASA Uncle would say what they were doing was impossible/too expensive/etc. They.
Principal scientist FANG friend of mine told me today that LLMs are overhyped and in two years no one will talk about them. I’d say we’re still so early but I think he might just be regarded idk.
12
14
264
Prediction: Elon Musk will have John Carmack be the head of his new AI efforts.
42
7
255
Anyone who thinks ChatGPT should be free & ad supported like Google is stuck in the wrong business model, repeating the last 20 years. Instead, ChatGPT & it’s offspring will be paid products with a monthly fee, like streaming services are. Inference is too expensive for just ads.
35
15
252
One of the most interest reveals at the Tesla AI Day today was how they are using innovations from language models to turn state handling when dealing with lane changes into a series of tokens for their self driving cars.
8
21
254
Sir Isaac Newton wrote his ground breaking Principia while fleeing the plague, waiting it out at his family farm while his University was closed. What are you going to do while waiting for the coronavirus to blow over? #NoPressure.
16
71
247
@astro_catherine Good for you for punching back at these bullies! Exciting to see what you do at your new NASA position.
0
0
207
Only in San Francisco would 5000 AI nerds just spontaneously get together to figure out what the heck just happened the last few weeks.
13
10
235
@_____Lightning No I agree they have a valid reason to go after him, it’s just looks really hypocritical happening the same day they choose not to press charges on the car jacking incident. If they were doing the rest of their job well it would be fine, but they aren’t.
26
3
224
I think the LLM bots are taking over Twitter, why are all of these essentially the same message?
49
14
225
@Der_Parrot The woman in the video appears to be fine and is still posting on Facebook, last post a few hours ago:
2
4
212
@theorizur I’ve been in tech for 25 years. The way you stay is to realize bad situations that aren’t working for you anymore and walk away, despite the money or status. I walked away from creating tech for Wall Street despite good money. I walked away from a career at Google when it wasn’t.
7
6
213
@CyrusMaher Sounds like they actually synthesized roughly 700 of them in a lab to confirm their chemical properties.
4
4
204
@wear_here @JaimeOrtega I lived in SF for 20 years, the last 5 years it has been very very broken. I was pro the progressive board ~2008 era as they actually got things done (first city to have 15 minimum wage for example). This current board are NIMBYs that fake being progressive in a cynical ploy to.
17
10
200
The new era of LLM assisted coding actually means those who have good communication skills will excel, as working effectively with an LLM at scale for coding means you have to be really good at communicating and giving feedback, just like working with an intern or co-worker.
20
21
198
If you’re an employee at Meta and hate the idea of VR, MR, and AR, why are you still there? I personally am excited about that direction & hold a contrarian view that XR will iterate into the future of the personal computer by the mid to late 2020s, but if you hate it why stay?.
42
5
194
Attention in Transformers scales quadratically in sequence length. Same lab that created FlashAttention exploring models that “hold the promise to have context lengths of millions… or maybe even a billion!”
1
36
194
Guy on Hacker News still arguing that "there's nothing to all this AI LLM hype".
21
0
183
People have ret-conned how 2007 iPhone was actually controversial & didn't support many basic things (no apps? no undo/redo? no physical keyboard? etc.). We see incredibly mature iPhone of 2023 & have forgotten how technology iteratively gets better & better from a v1 vision.
21
7
174