Eric Brachmann
@eric_brachmann
Followers
4,296
Following
605
Media
337
Statuses
1,904
Staff Scientist @NianticLabs . Throws #MachineLearning at traditional #ComputerVision pipelines to see what sticks. Differentiates the non-differentiable.
Europe
Joined January 2018
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
James Earl Jones
• 731475 Tweets
iPhone
• 588075 Tweets
Haitian
• 561136 Tweets
Tyreek
• 174547 Tweets
Cops
• 164220 Tweets
SIEMPRE FURIA SCAGLIONE
• 132886 Tweets
Jets
• 117763 Tweets
#WWERaw
• 107387 Tweets
49ers
• 95100 Tweets
Yunes
• 88714 Tweets
ZEE PRUK BIRTHDAY
• 69967 Tweets
#MartinFierro
• 43841 Tweets
シーザリオ
• 41181 Tweets
Aaron Rodgers
• 35250 Tweets
対象作品
• 23912 Tweets
Jordan Mason
• 19009 Tweets
ガチャチケット
• 18955 Tweets
Purdy
• 17955 Tweets
性の対象
• 17618 Tweets
キスィズ
• 16767 Tweets
Deebo
• 15611 Tweets
Saleh
• 15608 Tweets
Niners
• 14983 Tweets
Gunther
• 14410 Tweets
Aiyuk
• 13359 Tweets
小林邦昭さん
• 11699 Tweets
飛霄将軍
• 10808 Tweets
Last Seen Profiles
Pinned Tweet
📢A new learning-based approach to SfM:
#ACEZero
No img-to-img matching, optimises image-to-scene correspondences directly. Needs no pose priors. Works on unordered image sets. Efficiently handles thousands of images.
Paper:
Page:
12
87
475
✨ Presenting MicKey (
#CVPR2024
, Oral) ✨
We regress and match 3D camera coordinates rather then 2D key points, all in metric space. Gives you a scaled relative pose between two RGB images.
Paper:
Page:
8
63
451
A
#CVPR2023
Highlight✨:
♥️♠️ACE: Accelerated Coordinate Encoding♣️♦️
We learn implicit, neural maps in minutes that let us relocalize with SOTA accuracy.
📜:
💻:
📽️:
work by
@NianticLabs
Research
7
108
420
Code of
#ACEZero
is out.
A new approach to SfM. Learn the 3D scene without image-to-image matching. Naturally avoids the explosion of complexity for many images. ACE0 shines if you have dense coverage of a scene. Posing 10k images and more? Sure!
3
77
363
We release the first code package for NG-RANSAC: Estimating epipolar geometry from sparse correspondences. Implemented in
@PyTorch
and including various pre-trained models:
#ICCV2019
#ICCV19
#ICCV
#DeepLearning
#ComputerVision
#MachineLearning
4
74
253
We've published
@pytorch
code of Differentiable RANSAC for a toy problem: fitting lines. A CNN learns to predict points (middle) to which we robustly fit lines, trained end2end with DSAC. Right: A CNN which learns to predict line parameters directly. Code:
1
61
196
Read papers to write good papers.
Review papers to write great papers.
Read rebuttals to write good rebuttals.
Discuss rebuttals to write great rebuttals.
Great papers and great rebuttals are no guarantee to get accepted. Iterations are normal.
2
15
174
Does the "ground truth" treat you fairly? Watch this shocking
#ICCV21video
🤯🤯🤯 You will not believe what a 10-year-old baseline looks like today!
#ICCV2021
w/
@martinhu
@Carsten_Rother
@SattlerTorsten
9
23
157
Today, we launch Map-Free Visual Relocalisation. A twist on the usual relocalisation formula. A new dataset. A new benchmark.
Check out our video!
#MapFreeReloc
to be presented at
#ECCV2022
4
26
146
Niantic organises the first
#MapFreeReloc
workshop and challenge at
#ECCV2024
@eccvconf
!
We invite the submission of extended abstracts and papers. The challenge features two tracks, and includes a multi-frame version and $6000 in prizes.
Page:
3
30
139
We created some visualisations of MASt3R on the
#MapFreeReloc
dataset. The numbers do not lie, the results are amazing.
The public version is a bit worse than the private one which is top on the leaderboard. 0.751 AUC instead of 0.817.
Examples in 🧵including public vs private.
2
17
133
Major update and code release for our
#relocalization
pipeline, DSAC*
(Updated) paper:
Code:
Use RGB or RGB-D, train with an SfM model or a 3D scan. Train from images and poses alone and DSAC* discovers coarse geometry by itself:
3
20
124
We dedicate our oral presentation at
#CVPR2024
in Seattle to R2, who - in a great moment of integrity and reason - raised their initial "weak reject" rating to "strong reject" after rebuttal.
We hope they enjoy every minute of our 15 minute talk.
✨ Presenting MicKey (
#CVPR2024
, Oral) ✨
We regress and match 3D camera coordinates rather then 2D key points, all in metric space. Gives you a scaled relative pose between two RGB images.
Paper:
Page:
8
63
451
7
13
124
RGB image in, set of 3D primitives out. A
#CVPR2021
paper with
@florian_kluger
, H. Ackermann, M. Yang and B. Rosenhahn!
#ComputerVision
abs:
code:
We take RANSAC out of its comfort zone into scene understanding territory. 👇
2
19
123
I really like the Twitter format for presenting papers. So I thought I replicate that for my personal website (neglected for years...)
It's a simple static HTML, feel free to use as template.
4
22
111
Due to requests at
#ECCV2022
and to make our
#MapFreeReloc
dataset useful for more tasks, we make the SfM reconstructions of our train set publicly available.
🔥460 SfM models of outdoor scenes all around the world 🔥
Want to train 460 NeRFs? Go ahead.
6
20
113
Career update. I have come to chew bubble gum and do kick ass research. And I'm all out of bubble gum.
@NianticLabs
#80smoviesreferences
#augmentedreality
#
13
3
107
All talks of our
#ICCV2021
tutorial on visual localisation are available on YouTube.
In my talk about learning-based localisation, I discuss a challenge, a promise and a compromise.
Watch it here:
Links to all the amazing talks in 🧵👇
4
24
108
🔥Map-relative Pose Regression🔥(
#CVPR2024
highlight)
For years absolute pose regression did not work. There was some success by massively synthesising scene-specific data. We train scene-agnostic APR and it works.
Paper:
Page:
5
13
107
Third and last code package for NG-RANSAC (
#ICCV2019
) is online: NG-DSAC++ for camera re-localization, a re-implementation of DSAC++ in
@PyTorch
, extended with neural guidance.
Code:
Paper:
w/
@Carsten_Rother
@LabHeidelberg
1
27
104
Did you know there is a German word for working late into the night towards a deadline? Verschlimmbessern.
#CVPR2021
3
5
101
We are hiring! Want to work with me and an amazing team at
@NianticEng
on re-localisation at a global scale? Bring cutting-edge research into the hands of millions of users?
Consider applying for our Mapping and Localisation MLE role:
0
16
102
#ACEZero
is great to pose every frame of a video. But why not use SLAM? Sometimes it's advantageous to have no frame-to-frame assumption. No need to worry about loosing tracking.
In this example, we register "only" 70% of the frames. The rest is dismissed. Good or bad? Depends.
1
16
102
Niantic research has a strong presence at
#CVPR2023
with 5 papers (one highlight) and various contributions throughout workshops and tutorials.
Diffusion, NeRF, relocalisation, object pose, feature matching, depth and occlusions.
Here is where you can catch us:
1
14
100
We uploaded estimated poses of
#ACEZero
for the main paper experiments. If you do want to compare without running our code.
You also get the reconstruction videos, 78 in total.
Links in the README:
Eg reconstructing T&T Caterpillar from 11k images.
3
7
100
2 papers accepted to
#cvpr2020
🥳
Reinforced Feature Points: Use classic REINFORCE to optimize feature detection and description for the task you care about
CONSAC: fit multiple parametric models by learned sequential search (w/
@florian_kluger
)
More info soon!
#ComputerVision
7
7
98
#ACEZero
can be slower than other methods for sparse view reconstruction. That's a downside of the incremental learning approach. But methods can be combined.
Here we start from sparse poses of COLMAP, and then densify and refine with ACE0. ACE0 repairs drift along the way, too.
3
9
95
PhD student and looking for an
#internship
? Sink your teeth into juicy
#MachineLearning
and
#ComputerVision
problems right at the frontier of
#AugmentedReality
at
@NianticLabs
!
Spread the word! ❤️❤️❤️
0
31
88
When you try to solve difficult image pairs, it's important that you do not overshoot and start to hallucinate connections between unrelated images. The
#MapFreeReloc
benchmark checks for that. The inlier count of MicKey is pretty good in separating solvable and unsolvable cases.
2
7
89
.
@SattlerTorsten
talking about "Old School" methods at the
#ICCV2021
tutorial on visual localization. "Old" but not outdated!
Join us here:
I will talk about the "New School" later.
0
14
90
Reconstructing Ignatius (Tanks&Temples) from 7,843 images, starting from raw images without intrinsics, in one hour on a V100.
#ACEZero
1
9
89
Some motivation if you didn't make it into
#CVPR
:
- our first object coordinates paper: rejected from CVPR14, barely made it into ECCV14 (>250 citations)
- follow up: rejected from ICCV15, made it into CVPR16 (~200 citations)
- ESAC: rejected from CVPR19, made it into ICCV19
3
13
85
Catch Niantic Research at
#CVPR2024
!
- I will give a talk at the MonoDepth challenge, TUE-PM. Teaser below!
- Niantic presents two posters WED-PM: AirPlanes and MicKey. Do not miss the MicKey 📢Oral 📢 by
@axelbarrosotw
!
- One more poster FRI-AM: marepo (✨Highlight✨)
2
13
86
New retro wave of
#ComputerVision
: NG-RANSAC brings you the greatest hits of the 80s: RANSAC, multi-layer perceptrons, (classic) reinforcement learning. Lens flare for visualization of "cool", only. The paper:
#ICCV2019
#ICCV19
#ICCV
#DeepLearning
2
19
79
To
#CVPR2023
reviewers: Remember there are humans on the other end. Be strict in the matter but respectful in tone. Consider even being friendly in tone. Someone worked hard on this, and is proud. Don't bend but sweeten the pill.
2
8
80
With the
#ACErelocalizer
, we reduced mapping times from 15 hours to 5 minutes. You can imagine that further speed improvements are exponentially harder. Yet, we managed to squeeze out another 15% speed-up. Let me walk you through the steps:
1. Upgrade to
@pytorch
2.x
Fin.
A
#CVPR2023
Highlight✨:
♥️♠️ACE: Accelerated Coordinate Encoding♣️♦️
We learn implicit, neural maps in minutes that let us relocalize with SOTA accuracy.
📜:
💻:
📽️:
work by
@NianticLabs
Research
7
108
420
5
9
80
Automatic generation of ground truth is great but caution is advised.
Upcoming for
#ICCV2021
: For vis. relocalisation, we show that depending on how you generate GT, the ranking of relocalisers flips upside down:
@SattlerTorsten
@martinhu
@Carsten_Rother
4
15
78
Submitting relocalisation via pose regression to
#ICCV2023
? Rejected from
#CVPR2023
? Consider the
#MapFreeReloc
benchmark.
No need to beat DSAC*, hLoc, AS... they do not apply. This benchmark was created just for you 🫵
Also if you work on:
- features
- depth
- uncertainty
🧵
3
14
76
Unfortunately, none of the authors of this great paper was able to make it to
#CVPR2024
. I was able to put it up, but have to go to our other poster now. So let's give this poor poster some love on Twitter at least!
2
8
76
We uploaded alternative "ground truth" and full SfM models for the
#relocalisation
datasets 7Scenes and 12Scenes.
Working on reloc towards
#ECCV2022
and having trouble beating SOTA? The "ground truth" might play a role.
#ICCV2021
#betterlatethannever
1
22
73
Results of the current runner-up on the
#MapFreeReloc
leaderboard (single frame):
MicKey (
#CVPR2024
).
Code is available, might be a good starting point for the
#ECCV2024
map-free challenge!
2
10
73
When I first saw MASt3R on our
#MapFreeReloc
leaderboard, I thought someone was trolling us. The name did not help. After reading the paper and chatting to the authors, I believe it's legitimate.
What incredible progress on a new task in ~2 years. And more is possible, I'm sure!
Grounding Image Matching in 3D with MASt3R
@Vinc3nt_Leroy
Yohann Cabon
@JeromeRevaud
tl;dr: Dust3r with descriptor head and metric depth.
P.S. guys, why not some KD-tree, why new fast knn?
6
32
173
0
5
73
MAST3R leads the
#MapFreeReloc
leaderboard by a large margin. Glad that the code is out, so people can improve on it 💪 43 more days to compete in the
#MapFreeReloc
challenge for
#ECCV2024
!
The wait is over 📢 MAST3R is out! DUSt3R+ dense local feature maps & metric depth - 1st in
#MapFreeReloc
leaderboard, can handle 1000s of images 😀 !!
Blog:
Code:
Paper:
5
59
289
1
10
73
The moment has finally come. I tried to get the best education possible to prepare for this. Worked hard. Let's see whether it all payed off.
My son learned the word "why".
4
1
69
Personally, I find these results all the more remarkable considering that MicKey does not use any cross attention. Key points and descriptors are predicted per image, without considering the other view. Just vanilla matching and RANSAC after that.
Beyond providing metric estimates, MicKey can also deal with extreme view point changes, up to opposing shots. Here are a few examples of MicKey correspondences.
1
0
18
3
10
68
Stumbled across this
#ICCV2023
paper extending
#MapFreeReloc
to panoramic indoor views. Makes a lot of sense: A panorama serves as a one-shot map with wide coverage.
Calibrating Panoramic Depth Estimation for Practical Localization and Mapping, Kim et al
1
20
69
If you like these videos, we just pushed the code to generate them to the
#MapFreeReloc
repo. Might be handy for those who work towards the
#ECCV2024
challenge.
We created some visualisations of MASt3R on the
#MapFreeReloc
dataset. The numbers do not lie, the results are amazing.
The public version is a bit worse than the private one which is top on the leaderboard. 0.751 AUC instead of 0.817.
Examples in 🧵including public vs private.
2
17
133
2
11
67
Reinforced Feature Points, my first paper as last author, will be presented as an oral at
#CVPR2020
🥳🥳🥳
Congrats to
@AritraBhowmik6
!
arXiv:
1
15
68
#LightshipVPS
in action. Please note the alignment of the chain with the anchor top 🙀
@LightshipAR
@NianticEng
#VPS
#AR
#AugmentedReality
4
10
67
Learning-based visual relocalisation has made quite some progress over the years.
I'm giving an overview in our
#CVPR2023
tutorial on:
📢"Large-Scale Visual Localisation"
Mark June 19th in your calendar. Check out the full schedule of amazing talks:
3
16
67
Strange. My Twitter feed somehow suppresses all the tweets of PhD students from small labs that complain about the
#CVPR2022
social media motion. I see, however, plenty tweets by 1k+ follower accounts complaining that the former group will be at a disadvantage now 🙃
7
4
64
It's fun playing around with
#ACEZero
. Just point it to a set of images, and you are good to go.
(The Lego model? Yeah, I built that. Don't mention it. The middle section? Yeah, it can totally detach a stand-alone Ninja bike! Pretty cool? Well, if you say so... )
1
0
63
Bring me the horizon! I finally managed to prepare the code of NG-DSAC (
#ICCV2019
) for horizon line estimation.
Code:
Paper:
#deeplearning
#machinelearning
#computervision
1
14
62
Visual relocalization has been a bit of a niche topic of 3D vision. I hope this work can help it reaching a wider audience by showing how tightly coupled it is with reconstruction.
People think of reloc as a step after or within incremental SfM.
#ACEZero
turns that on its head.
📢A new learning-based approach to SfM:
#ACEZero
No img-to-img matching, optimises image-to-scene correspondences directly. Needs no pose priors. Works on unordered image sets. Efficiently handles thousands of images.
Paper:
Page:
12
87
475
0
3
62
You can use
#ACEZero
to refine a set of existing poses. In the example below, we start from KinectFusion poses. ACE0 repairs a defect in KF tracking, and the ACE0 refinement gains 4dB in PSNR for a Nerfacto model on top.
Everything tidied up in 8 minutes for 6000 frames 😇
3
10
62
What's
#MapFreeReloc
?
A learned model induces a scale-metric space, conditioned on a single reference frame. We localise new queries in that space.
I will talk about "Pose Estimation Beyond Feature Matching" on Monday morning at
#CVPR2023
at the Image Matching Workshop
#IMC2023
1
12
59
RANSAC vs. USAC vs. NG-RANSAC for essential matrix estimation from SIFT correspondences. Long (and slower) version:
1
10
57
I understand why ICCV/ECCV rank lower than CVPR because they are not annual. But why does ECCV rank lower than ICCV?
It would appear to me they recruit chairs and reviewers from roughly the same pool. ECCV is the second important deadline in even years, as ICCV is for odd years.
Top ten
#computervision
publication venues according to Google Scholar's annual "Top Publications"
1
11
65
18
3
56
Dear
#ECCV2024
reviewer,
"see weaknesses" is not a good "justification of rating". Every paper has weaknesses. The AC needs to understand why you think the weaknesses outweigh the strengths (or vice versa).
Best,
2
4
53
You would assume that since 2012
#DeepLearning
was applied to everything and their mother. But, as far as we know, CONSAC is the first learned multi-model robust estimator.
By
@florian_kluger
#CVPR2020
arXiv:
code:
2
18
51
Our call for research interns
@NianticEng
for 2022 is live! Interested in pushing the frontiers of AR? So are we. Work with us on a wide range of topics in the spectrum of
#ComputerVision
and
#MachineLearning
!
Interested? More info and application forms:
3
14
51
Today, Wednesday morning, at
#ICCV2019
, I will present NG-RANSAC at poster stand 143 (far end of the expo hall). I will try to explain the method on a simple toy problem, such that even I could understand it :) Come by, say hello!
#ICCV
#ICCV19
#DeepLearning
#MachineLearning
2
9
52
Two papers accepted to
@ICCV19
! Neural-Guided RANSAC (NG-RANSAC): A neural network guiding RANSAC data point selection, and Expert Sample Consensus (ESAC): An ensemble of scene coordinate experts for scalable camera re-localization.
#ICCV2019
#ComputerVision
#DeepLearning
2
8
51
I keep reading in papers that RANSAC is so excruciatingly slow that getting rid of it is a contribution.
Watch DSAC* estimating poses from thousands of correspondences in 30ms per frame, including network execution and RANSAC. 👇
Brrr.
3
3
51
Who said re-localisation in changing conditions is a problem?
4
1
50
Only 6650 actual submissions to
#CVPR2020
. End of the world prevented. (3350+ people were just curious what the current submission ID might be?)
1
17
50
@eccvconf
Oh, this is just cruel. Posting this and then it's still that 14 page, 1 column "aesthetic".
3
0
49
Extending on my earlier post about research trends at
#CVPR2018
, I wrote a little python script that plots topic popularity (measured by key word matches against paper titles) over time. It's a simple Jupyter notebook, so you can play around yourself:
1
25
48
There is this magical moment of excitement and joy, right after drafting a new approach, and before actually implementing it and seeing it fail in all the experiments.
3
3
49
Hopefully, someday I will review a paper that starts with:
"X is not a fundamental problem in computer vision. But we looked into it anyway."
3
0
48
Thanks
@ducha_aiki
for sharing our work faster than we could get it to arXiv. Its available there as well, now.
📜
TL;DR? Upload the gist of it into your brain via YouTube:
▶️
w/
@axelbarrosotw
,
@viprad
, Gabe, and
@dantkz
Two-view Geometry Scoring Without Correspondences
Axel Barroso-Laguna,
@eric_brachmann
Victor Adrian Prisacariu Gabriel Brostow
@dantkz
tl;dr: in title + nice analysis of the metrics for the epipolar geometry.I almost want to write a blogpost-review :)
2
20
85
2
14
46
I really like that I'm not able to rate "borderline" in the final
#ICCV2021
assessment. I'm split regarding several submissions, now I have to think twice as I have to lean in some direction. An unexpectedly enjoyable inner fight.
0
2
44
Turning regression into classification problems is one of the most unappealing workarounds we currently often resort to, I find.
6
3
45
How to explain the difficulties of hyper-parameter tuning to non-experts.
2
1
45
6/6
#CVPR2022
reviews done. Strong stack this one: worst score is borderline - never had this before. Innovative ideas and above-average writing. I'm impressed.
0
0
44
Finally, there is an extensive benchmark for 6D object pose estimation, presented at
#ECCV2018
. No learned method in top 3. Best learned method uses Random Forests instead of CNNs. Want to defend the honor of deep learning? It's a running competition ;)
2
26
42
Naver Labs is joining the
#MapFreeReloc
challenge as co-sponsor, giving $2000!
Thanks
@JeromeRevaud
and
@naverlabseurope
for making it happen!
We have a total of $6000 in prizes for the challenge winners.
0
4
44
The slides for my talk at the R6D workshop at
#iccv2019
are available at the workshop website:
A summary of our work on differentiating PnP, differentiable RANSAC, differentiable correspondence selection and differentiable expert selection.
#DeepLearning
1
10
43
Something I wanted to talk about so often to so many people was finally accepted at
#ICCV2021
1
1
41
A small thread on 3D rotations: Both log-quaternions (log-q) and axis-angles (aa) represent rotations with 3 parameters. But they are not the same, related by a factor of 2. The length of aa gives you the rotation angle, the length of log-q gives you half that angle.
2
8
38
Struggling to understanding SE(3)? You are certainly not alone. The thread below and all its replies collect a wonderful list of resources.
This summer I've been working to finally understand Lie Theory, the basis for proper estimation on over-parameterised manifolds like SE(3). There are some great tutorials for the roboticist out there; I especially like Micro Lie Theory by Solà et al.
13
109
634
0
8
40
On Wednesday I will be giving a talk at
@naverlabseurope
on visual localization. I will talk about differentiable RANSAC and new stuff, including learning to guide RANSAC and scalable re-localization. If you are in Grenoble, come by!
1
7
39
To add to my list of motivating failures: I had 2 submissions lined up for
#ECCV2020
. One stopped a few weeks ago, one stopped a few hours before deadline. So that happens, too!
At the same time, this is my contribution to
#slowscience
. You are welcome 😜
Some motivation if you didn't make it into
#CVPR
:
- our first object coordinates paper: rejected from CVPR14, barely made it into ECCV14 (>250 citations)
- follow up: rejected from ICCV15, made it into CVPR16 (~200 citations)
- ESAC: rejected from CVPR19, made it into ICCV19
3
13
85
0
0
39
Like so many others, I also had a lot of fun at
#CVPR2024
! It was great to chat with old friends and meet new people (Yes, we will release the ACE0 code soon!) 5 days of conference are still not enough to catch up with everybody... Hope to make up for some of that in Milano.
1
1
37
Sounds like something awesome is about to happen in the
#AugmentedReality
landscape. 😃 Visual reloc here and there, surely soonish everywhere.
0
3
37