Alberto Hojel
@AlbyHojel
Followers
3,366
Following
2,243
Media
230
Statuses
1,239
self-supervised vision and interpretability @berkeley_ai // ML for clouds @RainmakerCorp // expansionist
Berkeley, CA
Joined March 2020
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Brazil
• 960436 Tweets
Elon Musk
• 658004 Tweets
O Brasil
• 653507 Tweets
Chelsea
• 526347 Tweets
Alexandre de Moraes
• 469678 Tweets
Osimhen
• 281095 Tweets
Sancho
• 250066 Tweets
Sterling
• 211520 Tweets
Bluesky
• 178597 Tweets
Xandão
• 145112 Tweets
Politico
• 137848 Tweets
Napoli
• 135594 Tweets
Adeus Twitter
• 134207 Tweets
#مولد_ولي_العهد
• 104647 Tweets
Threads
• 97576 Tweets
Tchau
• 69685 Tweets
Labor Day
• 68256 Tweets
Eddie
• 64393 Tweets
Temple
• 58088 Tweets
VPNs
• 42449 Tweets
CONGRATULATIONS RM
• 39916 Tweets
#الجناح_مطلب_اهلاوي
• 31233 Tweets
CONGRATULATIONS NAMJOON
• 29878 Tweets
Michigan State
• 22145 Tweets
Oklahoma
• 18851 Tweets
ジェシー
• 18223 Tweets
Caitlin Clark
• 16194 Tweets
#ウルトラマンアーク
• 14893 Tweets
野菜の日
• 13053 Tweets
#شكرا_الوليد_بن_طلال
• 11702 Tweets
Shelton
• 11104 Tweets
Evades
• 10492 Tweets
Tiafoe
• 10116 Tweets
Pinned Tweet
My first paper!
How does the mysterious in-context learning of few-shot models arise? In our work, we peel back the veil of visual prompting models and open the black box!
Inside these models, we actually find internal representations of the tasks implicitly described by the
14
15
193
Day 7,583 (since birth):
progress: solid
coffees: 4 espresso shots
defense hackathons signed up to: 1 more (this weekend)
Cooking up something new. Taking my whack at PEFT, lets see what all these LLM ppl are getting excited about.
Peeked into the HF Llama2 caching
18
14
485
Tim Darcet was looking through DiNOv2 (a self supervised vision model used for deep feature extraction) and saw something strange happening...
A dive into one of the recent ICLR outstanding paper award recipients: "Vision Transformers Need Registers" 🧵
8
63
448
there are cathedrals everywhere for those with the eyes to see
5
29
406
i woke up this January 1st knowing exactly what my New Year’s resolution would be: i wanted to publish a paper.
ever since I was a kid, I dreamed of becoming a machine learning researcher. I wanted to be a scientist.
When I got to Berkeley, my dreams were quickly overshadowed
13
8
249
Tim Brooks and Bill Peebles from OpenAI came to my Deep Unsupervised Learning class to talk about video gen.
Can’t hear much (iPhone microphones kinda suck) but here’s a snippet where he talks about emergent 3D consistency without any inductive bias to promote it.
11
26
207
Our work "Finding Visual Task Vectors" has been accepted to
#ECCV2024
!🎉Big thanks to co-authors
@YutongBAI1002
@trevordarrell
@amirgloberson
@_amirbar
!
This will be my first ECCV 😀
We will also be presenting at the Mechanistic Interpretability Workshop ICML 2024!
Finding Visual Task Vectors
Find task vectors, activations that encode task-specific information, which guide the model towards performing a task better than the original model w/o the need for input-output examples
5
68
347
6
26
174
display your work even if no one is looking
work on your craft even if it doesn’t call for attention
the right people will start noticing you
and you just keep getting better each time
7
7
163
Day 7,584 (since birth):
"articulate your dreams"
cloud microphysics textbooks bought: 2 ($213; someone bless me with a grant for textbooks 🙏)
wild sightings of
@AravSrinivas
: 1
Got my commonsense datasets, ready to run ReFT. Let's see how long it takes us to beat it.
9
3
157
Built a semantic search engine for 3D space based on LERF!
Day 7,585 - 7,588:
- Hackathon at Shack15
- Built a 3D search engine based on LERF
- Imagine RAG but for Spatial Analysis
- The seeds of 3D understanding
Shoutout to
@chesterzelaya
who's building the hardware to make this scalable, and to our teammates
@pfelgueres
and Ivan!
2
0
25
4
1
107
Can the recent advances in volumetric representations prove valuable in battle environments? This is what I set out to answer at the El Segundo Defense hackathon.
11
5
99
What 3D researchers will go through to collect nice data will always surprise me
11
5
94
Scale vs Architecture
Question: if you want to build a simulation engine using a video gen model, should you incorporate inductive biases that promote 3D consistency?
Bill from the Sora team’s answer: inductive biases always come back to bite you. Scale is enough.
4
13
79
Brownsville's S-band radar caught a starship launch last year (green blob)
0
3
77
What the actual f
This is actually hilarious, I love it
Had to share
6
4
75
The universe is a chaotic system. To me it is unclear if humanity can converge to a stable equilibrium. You either progress or you regress. Accelerate or cease. One thing is clear to me though: life is an optimization machine and will continue to select for organisms that
5
11
70
Too many books to read and not enough time. Happy to receive recommendations…
19
1
64
DM that researcher who just released a cool project. You'll be surprised how much you can learn and how nice people are.
~They~ don't want you to know researchers love to talk about their research.
5
0
53
At some point you need to start “doing the math” instead of reading it
4
4
49
It’s become increasingly clear to me the importance of rizz and ambition for willing a deep tech startup into existence from mere proof of concept research papers
1
0
48
Lots of people saying "no excuses, do both."
My hot take is that you only really hit your full potential if you focus on 1 thing and give it your all.
Compound growth.
6
0
47
First impressions of the gundo: wow.
Thanks
@ADoricko
@rasmusdeymeyer
@isaiah_p_taylor
@soren_ma
and everyone involved!
1
2
40
You've heard of sim2 real, but what about real2sim?
evaluating policies trained on real data can be time consuming and the set up must be perfect (no bumping into cameras, and must be easily reproducible).
what if one could simply evaluate inside of a simulated environment?
1
2
40
Bill from the Sora team at Open AI on the satisfying 3D coherency of their video gen model as proven by running it through a NeRF pipeline:
1
5
38
Everyday gets easier but you have to do it everyday that’s the hard part
1
3
36
By far the most cracked team I’ve ever met working on the most ambitious problem I’ve had the pleasure to learn about. Come join us in making earth habitable and bringing forth an era of abundance.
WANT TO MAKE RAIN?
Want to develop arcane hardware in the depths of the desert and at the edges of the Earth?
Rainmaker is hiring flight test engineers, signal processing engineers, MechEs & more
Apply below for adventure and hardship 👇
9
20
145
4
0
36
Code Release Alert!🚨🚨
We release our implementation of REINFORCE for "Finding Visual Task Vectors" in the activation space of MAE-VQGAN.
We show it improves model performance and even amortizes computation by 25% percent!
More below!🧵
1
7
35
First week in the gundo, already got to meet one of my heroes alongside the highest concentration of raw intelligence, ambition, and dedication. It’s gonna be a good summer.
1
1
36
The cool thing about experimentalists is that you kinda don’t have to fully understand what you’re doing just figure out what works and keep going in that direction.
1
0
33
I’m telling you, 3D is coming and it’s coming fast
With OpenMask3D, you can search 3D scenes directly via free-form text queries. Drop by the
#ICLR2024
Google booth today at 9:30am for a demo with
@FrancisEngelman
on using visual-language models to achieve open-vocabulary 3D instance segmentation.
8
91
413
2
2
34
I believe this is in part due to the fact that deep learning is such a young field. A big reason why it’s a relatively easier field to break into and start pushing the needle forward (the fact that is so new and that ML ppl usually dont know the historical context). Many times
Mel Andrews (
@bayesianboy
): “The general rule is that people in machine learning do not read”.
🔥
42
163
1K
1
2
31
If you’re feeling defeated and losing momentum on a problem, put on the Oppenheimer’s soundtrack (real) and feel the intelligence flow through your ears into your brain (scientific fact).
4
3
32
Day 7,589:
Had to lock in for my Philosophy of Mathematics final essay
TLDR: How can we measure infinite sets? How can we determine if one infinity is greater than another? Finally, if we have two different sizes of infinity how do we know if there is a size that falls
2
0
32
So that's what he did, and turns out it not only improves model performance but it even makes the attention maps more interpretable as there are no more high-norm tokens standing out!
1
2
31
i am limited by the technology of my time (waiting for conda environment to solve)
2
1
28
@ADoricko
. Rapidly deployable 8ft-wingspan fixed-wing cloud seeding drones + a mobile Doppler on Wheels (DoW) truck-borne radar fleet.
Water abundance via weather modification.
Very much theory only gets you so far / rapid iteration vibe
Thrilled to join the team this summer.
0
3
26
You know how models are just big matrix multiplications where you combine your weights with your data? You can literally line up all of your weights into one super high dimensional vector (7billion dimensions if the model is 7B) and boom that vector describes that the position of
What can you do with this? Turns out a bunch of stuff and its pretty handy like...
> forgetting via negation
> getting better at new tasks
> and combining them via analogies
5
0
12
1
0
28
RADAR is so complex. Just look at what they have to do to take into account attenuation from your radome.
Robot arm + reflectometer
2
0
25
Day 7,585 - 7,588:
- Hackathon at Shack15
- Built a 3D search engine based on LERF
- Imagine RAG but for Spatial Analysis
- The seeds of 3D understanding
Shoutout to
@chesterzelaya
who's building the hardware to make this scalable, and to our teammates
@pfelgueres
and Ivan!
2
0
25
1,000 femtosats to explore the solar system for $5 million dollars. There's ~30,000 asteroids near earth, but we've only visited ~20 and photographed ~45.
Solar sails have been dreamt about since the 1930s; nanotech and MEMS is finally making it possible.
@chesterzelaya
1
5
24
@CJHandmer
You’re actually a great person to ask. Do PhDs level up your ability to reach through the tech skill tree and in general set you up to be able to tackle harder problems in your startup? Or should you just go tackle hard problems and learn by doing?
0
1
23
During training, let us include a handful of extra learnable tokens so that the model can use them without having to use image patches for computation.
At the end of your forward pass, you just discard them. With this, you are allowing the model to perform more complex
2
2
23
Let’s assign semantic meaning (text embeddings) to squishy blobs of color (Gaussians) and solve 3D scene understanding! Update soon
1
0
22
If you don’t have a caffeine addiction + a vertical monitor, ngmi.
2
1
21
Sora thread continued.
Is distribution shift in captions used to train vs those provided by a user an issue? Special work goes into making sure almost everything is “in-distribution”
Basically the method engineered by DALLE 3 where gpt rewrites the prompt, is central to Sora
1
1
21
Classic Goodharts Law
0
0
21
Curious how sensitive weather is to perturbations? Smoke plumes from wildfires can blossom into towering pyrocumulus clouds and can even turn into full fledged thunderstorms. It’s amazing how fast the cloud appears…
3
1
21
@IntuitMachine
The beginning of infinity by David Deutsch seeks to answer this! TLDR, the invention of the scientific method.
1
0
19
Sometimes not knowing the full extent of a problem can be a great advantage.
Sometimes you need to cut your lit review short and just start tryinh
If you keep reading about how it’s impossible to do the impossible thing you may start to believe it is indeed impossible
2
0
20
Whaat! 4D monocular is on track to being solved?
Using segmentation and in-painting models, go from a single video to a 4D GSplat of the environment.
Will spatial video become the new norm in the near future? Hopefully apple builds this into the next iphone.
1
0
20
Adversarial attacks as a service is something I had not internalized. Artists fighting back against Gen AI by poisoning their personal storage on Adobe Cloud. What a wild timeline
I don't use Adobe's Creative Cloud because I like keeping my files locally.
But seeing how they think they can use anything you upload there for AI training purposes, I'm filling up my entire 20 GB storage with random Nightshade-poisoned images.
I encourage others do the same!
602
10K
61K
2
0
20
A primer on synthetic data for computer vision…
It's pretty obvious now that CV can unlock new areas for automation such as industrial quality checking, self driving, and general inspection. But collecting and annotating data can be a laborious process.🧵
2
1
19
But what was Tim Darcet doing and why do vision models need registers as his new paper claims?
Well he was looking through the activations of the model trying to find potential areas for improvement when he noticed that sometimes the messages getting sent between tokens by the
1
4
18
Wanna train RL Surgeons to perform brain surgery on your in-context learning model to enforce task-following and even improve task performance with a small amount of samples? Look no further than our repo that just hit 25 stars (woohoo!).
2
0
18
Era of 3D Vision incoming. Get ready.
We quietly dropped a cloud-based NeRF service a few weeks ago, so now everyone has the ability to produce super high-quality radiance fields with Google Cloud. Check out this great
@RadianceFields
blog post to learn more:
4
22
155
1
1
18
I learned about hail spikes today. Blew my mind.
Weather radar works by emitting energy in a beam off into the atmosphere and sort of seeing what bounces back and how long it took to bounce back.
Hail spikes are artifacts you get from the energy bouncing on the hail, bouncing
4
0
17
Anyone over 10 is cooked (I wish I had this when I was a kid). They’ll be brilliant.
Marc Andreessen recently tweeted “In our new era of AI: Every child will have an A.I. tutor that is infinitely patient, infinitely knowledgeable, infinitely helpful.”
That era is here.
Try it yourself 👉
1/n
249
1K
6K
3
0
16
How steerable is Sora? What controls do researchers have?
1) interpolation
2) zero-shot video to video (finetuning?)
1
1
16
No experts on frontiers, only pioneers.
Ppl don’t want u to know the playing field levels at the edge of knowledge.
1
1
16
What the image above shows is that some tokens that correspond to areas of low information are behaving strangely. In the conversation analogy, they are too talkative!
What could this be? Tim wondered.
He hypothesized that these tokens were being used as registers to perform
1
0
15
Nothing more satisfying than a paper author answering your inquiry email within 20 minutes!
1
1
15
The point of foundation models is that they’re trained on internet-scale datasets. If you just have a regular dataset then it’s just sparkling machine learning.
1
1
15
Special thanks to the Entrepreneur First, Apollo Defense team, the judges, and everyone involved!
@the_simonpastor
@tommyjtietjen
@join_ef
@apollo_defense
@johncoogan
@8vc
@Andercot
@AtomsNotBits
@NateSalander
@anishk__
@rasmusdeymeyer
2
1
15
THis is beautiful
Over the last 4 months, we've partnered closely with
@RainmakerCorp
and
@ADoricko
to create a new aesthetic for technology companies. A brand and website that communicates Rainmaker's mission to Make Earth Habitable. Let's dive in 🧵
7
13
126
0
0
14
Nothing quite like excessive caffeine consumption while zooming inside a metal cylinder with slow wifi
New ideas emerge
3
0
14
Are you working on an a thing you want to make your whole life?
0
0
17
Can it handle indoor scenes like bodycam footage or head mounted cameras? Honestly, for vanilla gaussian splatting this is pretty solid IMO.
1
0
13