Eric Brachmann
@eric_brachmann
Followers
5K
Following
2K
Media
357
Statuses
2K
Account dormant. Consider reaching out on other platforms.
Europe
Joined January 2018
📢A new learning-based approach to SfM: #ACEZero. No img-to-img matching, optimises image-to-scene correspondences directly. Needs no pose priors. Works on unordered image sets. Efficiently handles thousands of images. Paper: Page:
13
92
485
✨ Presenting MicKey (#CVPR2024, Oral) ✨. We regress and match 3D camera coordinates rather then 2D key points, all in metric space. Gives you a scaled relative pose between two RGB images. Paper: Page:
8
63
442
A #CVPR2023 Highlight✨:. ♥️♠️ACE: Accelerated Coordinate Encoding♣️♦️ . We learn implicit, neural maps in minutes that let us relocalize with SOTA accuracy. 📜: 💻: 📽️: work by @NianticLabs Research
7
107
413
Code of #ACEZero is out. A new approach to SfM. Learn the 3D scene without image-to-image matching. Naturally avoids the explosion of complexity for many images. ACE0 shines if you have dense coverage of a scene. Posing 10k images and more? Sure!
3
78
357
We release the first code package for NG-RANSAC: Estimating epipolar geometry from sparse correspondences. Implemented in @PyTorch and including various pre-trained models: #ICCV2019 #ICCV19 #ICCV #DeepLearning #ComputerVision #MachineLearning
4
74
246
We've published @pytorch code of Differentiable RANSAC for a toy problem: fitting lines. A CNN learns to predict points (middle) to which we robustly fit lines, trained end2end with DSAC. Right: A CNN which learns to predict line parameters directly. Code:
1
61
193
We wrote a text on how we envision the future of visual mapping: So far, we have trained ~50 million ACE networks. 😎 Together, that's 150 trillion parameters. Not bad! But the future lies in going from local small models to a global large model.
3
33
197
Read papers to write good papers. Review papers to write great papers. Read rebuttals to write good rebuttals. Discuss rebuttals to write great rebuttals. Great papers and great rebuttals are no guarantee to get accepted. Iterations are normal.
2
15
172
Does the "ground truth" treat you fairly? Watch this shocking #ICCV21video 🤯🤯🤯 You will not believe what a 10-year-old baseline looks like today!. #ICCV2021 w/ @martinhu @Carsten_Rother @SattlerTorsten
9
22
153
After all these years, I still enjoy submitting to conferences. Is it polishing text late at night? Arguing with reviewers? Adding another publication to the pile? No. It's because each submission is a testimony, an 8-page certificate, that I learned something new. #CVPR2025.
0
7
154
Today, we launch Map-Free Visual Relocalisation. A twist on the usual relocalisation formula. A new dataset. A new benchmark. Check out our video! #MapFreeReloc to be presented at #ECCV2022
4
26
141
Niantic organises the first #MapFreeReloc workshop and challenge at #ECCV2024 @eccvconf!. We invite the submission of extended abstracts and papers. The challenge features two tracks, and includes a multi-frame version and $6000 in prizes. Page:
3
30
139
We created some visualisations of MASt3R on the #MapFreeReloc dataset. The numbers do not lie, the results are amazing. The public version is a bit worse than the private one which is top on the leaderboard. 0.751 AUC instead of 0.817. Examples in 🧵including public vs private.
2
17
131
Ahead of #ECCV2024, we held the #MapFreeReloc challenge. The winner is . 🥁. 🥇MASt3R. Congrats to @Vinc3nt_Leroy, Yohann, @JeromeRevaud for first place! . No-one was able to beat MASt3R throughout the challenge duration. Nevertheless, we had amazing further submissions! 👇🧵
2
27
128
Major update and code release for our #relocalization pipeline, DSAC*. (Updated) paper: Code: Use RGB or RGB-D, train with an SfM model or a 3D scan. Train from images and poses alone and DSAC* discovers coarse geometry by itself:
3
20
122
We dedicate our oral presentation at #CVPR2024 in Seattle to R2, who - in a great moment of integrity and reason - raised their initial "weak reject" rating to "strong reject" after rebuttal. We hope they enjoy every minute of our 15 minute talk.
✨ Presenting MicKey (#CVPR2024, Oral) ✨. We regress and match 3D camera coordinates rather then 2D key points, all in metric space. Gives you a scaled relative pose between two RGB images. Paper: Page:
6
13
120
RGB image in, set of 3D primitives out. A #CVPR2021 paper with @florian_kluger, H. Ackermann, M. Yang and B. Rosenhahn! #ComputerVision. abs: code: We take RANSAC out of its comfort zone into scene understanding territory. 👇
2
19
121
I really like the Twitter format for presenting papers. So I thought I replicate that for my personal website (neglected for years. ). It's a simple static HTML, feel free to use as template.
4
22
111
Due to requests at #ECCV2022 and to make our #MapFreeReloc dataset useful for more tasks, we make the SfM reconstructions of our train set publicly available. 🔥460 SfM models of outdoor scenes all around the world 🔥. Want to train 460 NeRFs? Go ahead.
6
20
110
The main output of #ACEZero are image poses. But it also produces a point cloud. Not great in itself as it's just a visualisation of the implicit scene representation - but good enough to initialise Gaussian splats. ACE0 + Splatfacto 👇
1
10
109
We updated the #ACEZero repository to make it compatible with newer versions of @nerfstudioteam. We added some functions to make it easy to train Gaussian splats on top of ACE0. See our FAQ: "I want Gaussian splats from my images. What do I need to do?".
0
18
109
All talks of our #ICCV2021 tutorial on visual localisation are available on YouTube. In my talk about learning-based localisation, I discuss a challenge, a promise and a compromise. Watch it here: Links to all the amazing talks in 🧵👇
4
24
105
Career update. I have come to chew bubble gum and do kick ass research. And I'm all out of bubble gum. @NianticLabs #80smoviesreferences #augmentedreality #
13
3
105
🔥Map-relative Pose Regression🔥(#CVPR2024 highlight) . For years absolute pose regression did not work. There was some success by massively synthesising scene-specific data. We train scene-agnostic APR and it works. Paper: Page:
5
12
106
Third and last code package for NG-RANSAC (#ICCV2019) is online: NG-DSAC++ for camera re-localization, a re-implementation of DSAC++ in @PyTorch, extended with neural guidance. Code: Paper: w/ @Carsten_Rother @LabHeidelberg
1
27
100
#ACEZero is great to pose every frame of a video. But why not use SLAM? Sometimes it's advantageous to have no frame-to-frame assumption. No need to worry about loosing tracking. In this example, we register "only" 70% of the frames. The rest is dismissed. Good or bad? Depends.
1
16
103
I now joined the group of people making videos of random objects wherever I am to see whether it reconstructs. (#ACEZero + Splatfacto)
1
11
98
Did you know there is a German word for working late into the night towards a deadline? Verschlimmbessern. #CVPR2021.
3
5
101
We are hiring! Want to work with me and an amazing team at @NianticEng on re-localisation at a global scale? Bring cutting-edge research into the hands of millions of users?. Consider applying for our Mapping and Localisation MLE role:.
0
16
99
Niantic research has a strong presence at #CVPR2023 with 5 papers (one highlight) and various contributions throughout workshops and tutorials. Diffusion, NeRF, relocalisation, object pose, feature matching, depth and occlusions. Here is where you can catch us:.
1
14
100
We uploaded estimated poses of #ACEZero for the main paper experiments. If you do want to compare without running our code. You also get the reconstruction videos, 78 in total. Links in the README: Eg reconstructing T&T Caterpillar from 11k images.
3
7
99
2 papers accepted to #cvpr2020 🥳. Reinforced Feature Points: Use classic REINFORCE to optimize feature detection and description for the task you care about. CONSAC: fit multiple parametric models by learned sequential search (w/ @florian_kluger). More info soon! #ComputerVision.
7
7
98
#ACEZero can be slower than other methods for sparse view reconstruction. That's a downside of the incremental learning approach. But methods can be combined. Here we start from sparse poses of COLMAP, and then densify and refine with ACE0. ACE0 repairs drift along the way, too.
2
8
94
As you write your #CVPR2025 papers, think of captivating first sentences. Example? . "Creating noise from data is easy; creating data from noise is generative modeling.". (from Song et al., ICLR21, .
3
12
92
Reconstructing Ignatius (Tanks&Temples) from 7,843 images, starting from raw images without intrinsics, in one hour on a V100. #ACEZero
1
9
89
PhD student and looking for an #internship? Sink your teeth into juicy #MachineLearning and #ComputerVision problems right at the frontier of #AugmentedReality at @NianticLabs!. Spread the word! ❤️❤️❤️.
0
31
86
.@SattlerTorsten talking about "Old School" methods at the #ICCV2021 tutorial on visual localization. "Old" but not outdated!. Join us here: I will talk about the "New School" later.
0
14
88
When you try to solve difficult image pairs, it's important that you do not overshoot and start to hallucinate connections between unrelated images. The #MapFreeReloc benchmark checks for that. The inlier count of MicKey is pretty good in separating solvable and unsolvable cases.
2
7
88
Dust3r takes over from RoMa as the leading method in #MapFreeReloc!
done, DUSt3R now first of the leaderboard!.
3
11
87
Catch Niantic Research at #CVPR2024!. - I will give a talk at the MonoDepth challenge, TUE-PM. Teaser below!. - Niantic presents two posters WED-PM: AirPlanes and MicKey. Do not miss the MicKey 📢Oral 📢 by @axelbarrosotw!. - One more poster FRI-AM: marepo (✨Highlight✨)
2
13
86
Some motivation if you didn't make it into #CVPR:. - our first object coordinates paper: rejected from CVPR14, barely made it into ECCV14 (>250 citations). - follow up: rejected from ICCV15, made it into CVPR16 (~200 citations). - ESAC: rejected from CVPR19, made it into ICCV19.
3
13
84
New retro wave of #ComputerVision: NG-RANSAC brings you the greatest hits of the 80s: RANSAC, multi-layer perceptrons, (classic) reinforcement learning. Lens flare for visualization of "cool", only. The paper: #ICCV2019 #ICCV19 #ICCV #DeepLearning
2
19
76
To #CVPR2023 reviewers: Remember there are humans on the other end. Be strict in the matter but respectful in tone. Consider even being friendly in tone. Someone worked hard on this, and is proud. Don't bend but sweeten the pill.
2
8
78
With the #ACErelocalizer, we reduced mapping times from 15 hours to 5 minutes. You can imagine that further speed improvements are exponentially harder. Yet, we managed to squeeze out another 15% speed-up. Let me walk you through the steps:. 1. Upgrade to @pytorch 2.x. Fin.
A #CVPR2023 Highlight✨:. ♥️♠️ACE: Accelerated Coordinate Encoding♣️♦️ . We learn implicit, neural maps in minutes that let us relocalize with SOTA accuracy. 📜: 💻: 📽️: work by @NianticLabs Research
5
9
81
Automatic generation of ground truth is great but caution is advised. Upcoming for #ICCV2021: For vis. relocalisation, we show that depending on how you generate GT, the ranking of relocalisers flips upside down:. @SattlerTorsten @martinhu @Carsten_Rother
4
15
76
Submitting relocalisation via pose regression to #ICCV2023? Rejected from #CVPR2023? Consider the #MapFreeReloc benchmark. No need to beat DSAC*, hLoc, AS. they do not apply. This benchmark was created just for you 🫵. Also if you work on:.- features.- depth.- uncertainty .🧵
3
14
73
Unfortunately, none of the authors of this great paper was able to make it to #CVPR2024. I was able to put it up, but have to go to our other poster now. So let's give this poor poster some love on Twitter at least!
2
8
75
The #MapFreeReloc workshop next Monday at #ECCV2024 has such an incredible lineup of speakers. Jakob (@jajuengel), Vincent (@Vinc3nt_Leroy), Simon (@SimonLynen), Torsten (@SattlerTorsten), Shubham (@shubhtuls) and Victor (@viprad)!. Schedule:
0
12
73
We uploaded alternative "ground truth" and full SfM models for the #relocalisation datasets 7Scenes and 12Scenes. Working on reloc towards #ECCV2022 and having trouble beating SOTA? The "ground truth" might play a role. #ICCV2021 #betterlatethannever
1
22
71
Results of the current runner-up on the #MapFreeReloc leaderboard (single frame): . MicKey (#CVPR2024). Code is available, might be a good starting point for the #ECCV2024 map-free challenge!.
2
10
72
MAST3R leads the #MapFreeReloc leaderboard by a large margin. Glad that the code is out, so people can improve on it 💪 43 more days to compete in the #MapFreeReloc challenge for #ECCV2024!.
The wait is over 📢 MAST3R is out! DUSt3R+ dense local feature maps & metric depth - 1st in #MapFreeReloc leaderboard, can handle 1000s of images 😀 !!.Blog: .Code: Paper:
1
10
71
When I first saw MASt3R on our #MapFreeReloc leaderboard, I thought someone was trolling us. The name did not help. After reading the paper and chatting to the authors, I believe it's legitimate. What incredible progress on a new task in ~2 years. And more is possible, I'm sure!.
Grounding Image Matching in 3D with MASt3R. @Vinc3nt_Leroy Yohann Cabon @JeromeRevaud . tl;dr: Dust3r with descriptor head and metric depth. P.S. guys, why not some KD-tree, why new fast knn?.
0
5
73
The moment has finally come. I tried to get the best education possible to prepare for this. Worked hard. Let's see whether it all payed off. My son learned the word "why".
4
1
67
Personally, I find these results all the more remarkable considering that MicKey does not use any cross attention. Key points and descriptors are predicted per image, without considering the other view. Just vanilla matching and RANSAC after that.
Beyond providing metric estimates, MicKey can also deal with extreme view point changes, up to opposing shots. Here are a few examples of MicKey correspondences.
3
10
68
Stumbled across this #ICCV2023 paper extending #MapFreeReloc to panoramic indoor views. Makes a lot of sense: A panorama serves as a one-shot map with wide coverage. Calibrating Panoramic Depth Estimation for Practical Localization and Mapping, Kim et al.
1
20
68
If you like these videos, we just pushed the code to generate them to the #MapFreeReloc repo. Might be handy for those who work towards the #ECCV2024 challenge.
We created some visualisations of MASt3R on the #MapFreeReloc dataset. The numbers do not lie, the results are amazing. The public version is a bit worse than the private one which is top on the leaderboard. 0.751 AUC instead of 0.817. Examples in 🧵including public vs private.
2
11
65
Reinforced Feature Points, my first paper as last author, will be presented as an oral at #CVPR2020 🥳🥳🥳. Congrats to @AritraBhowmik6! . arXiv:
1
15
67
#LightshipVPS in action. Please note the alignment of the chain with the anchor top 🙀. @LightshipAR @NianticEng #VPS #AR #AugmentedReality
4
10
63
Learning-based visual relocalisation has made quite some progress over the years. I'm giving an overview in our #CVPR2023 tutorial on: .📢"Large-Scale Visual Localisation". Mark June 19th in your calendar. Check out the full schedule of amazing talks:.
3
16
63
Strange. My Twitter feed somehow suppresses all the tweets of PhD students from small labs that complain about the #CVPR2022 social media motion. I see, however, plenty tweets by 1k+ follower accounts complaining that the former group will be at a disadvantage now 🙃.
7
4
63
Catching the last days of summer in Saxon Switzerland. Filming random stuff. (#ACEZero + Splatfacto)
4
7
65
It's fun playing around with #ACEZero. Just point it to a set of images, and you are good to go. (The Lego model? Yeah, I built that. Don't mention it. The middle section? Yeah, it can totally detach a stand-alone Ninja bike! Pretty cool? Well, if you say so. )
1
0
63
Bring me the horizon! I finally managed to prepare the code of NG-DSAC (#ICCV2019) for horizon line estimation. Code: Paper: #deeplearning #machinelearning #computervision
1
14
59
You can use #ACEZero to refine a set of existing poses. In the example below, we start from KinectFusion poses. ACE0 repairs a defect in KF tracking, and the ACE0 refinement gains 4dB in PSNR for a Nerfacto model on top. Everything tidied up in 8 minutes for 6000 frames 😇
3
10
61
Visual relocalization has been a bit of a niche topic of 3D vision. I hope this work can help it reaching a wider audience by showing how tightly coupled it is with reconstruction. People think of reloc as a step after or within incremental SfM. #ACEZero turns that on its head.
📢A new learning-based approach to SfM: #ACEZero. No img-to-img matching, optimises image-to-scene correspondences directly. Needs no pose priors. Works on unordered image sets. Efficiently handles thousands of images. Paper: Page:
0
3
60
Niantic is looking for research interns for next summer in London. Interested in the next generation of reconstruction, mapping and visual relocalization? Apply! (And send me a DM for good measure). Let's explore what the bitter-sweet lesson can do for 3D vision. Link below.
1
9
66
What's #MapFreeReloc?. A learned model induces a scale-metric space, conditioned on a single reference frame. We localise new queries in that space. I will talk about "Pose Estimation Beyond Feature Matching" on Monday morning at #CVPR2023 at the Image Matching Workshop #IMC2023
1
12
59
RANSAC vs. USAC vs. NG-RANSAC for essential matrix estimation from SIFT correspondences. Long (and slower) version:
1
10
55
I understand why ICCV/ECCV rank lower than CVPR because they are not annual. But why does ECCV rank lower than ICCV?. It would appear to me they recruit chairs and reviewers from roughly the same pool. ECCV is the second important deadline in even years, as ICCV is for odd years.
18
3
54
#ACEZero will be presented in pretty much exactly one week in oral session 2C at #ECCV2024 in Milano. Try the code now, and roast me with live questions! 🫣.
📢A new learning-based approach to SfM: #ACEZero. No img-to-img matching, optimises image-to-scene correspondences directly. Needs no pose priors. Works on unordered image sets. Efficiently handles thousands of images. Paper: Page:
1
3
53
You would assume that since 2012 #DeepLearning was applied to everything and their mother. But, as far as we know, CONSAC is the first learned multi-model robust estimator. By @florian_kluger #CVPR2020. arXiv: code:
2
17
51
Our call for research interns @NianticEng for 2022 is live! Interested in pushing the frontiers of AR? So are we. Work with us on a wide range of topics in the spectrum of #ComputerVision and #MachineLearning!. Interested? More info and application forms:.
3
14
51
Are you looking forward to the presentation of MASt3R at #ECCV2024? . Let me tell you a secret 🤫🤫🤫 MASt3R gets covered in an extended talk at the #MapFreeReloc workshop by @Vinc3nt_Leroy on Mon, and a poster too! Skip the line of the main conference, thank me later.
And of course we'll also be there for the awesome #MapFreeReloc Workshop. This task really is the future: blending DL methods for scale and traditional relative pose estimation!. And good news, there's still room for improvements. @JeromeRevaud @naverlabseurope @eric_brachmann
1
12
51
Today, Wednesday morning, at #ICCV2019, I will present NG-RANSAC at poster stand 143 (far end of the expo hall). I will try to explain the method on a simple toy problem, such that even I could understand it :) Come by, say hello! #ICCV #ICCV19 #DeepLearning #MachineLearning
2
9
50
Dear #ECCV2024 reviewer,."see weaknesses" is not a good "justification of rating". Every paper has weaknesses. The AC needs to understand why you think the weaknesses outweigh the strengths (or vice versa). Best,.
2
4
52
Two papers accepted to @ICCV19! Neural-Guided RANSAC (NG-RANSAC): A neural network guiding RANSAC data point selection, and Expert Sample Consensus (ESAC): An ensemble of scene coordinate experts for scalable camera re-localization. #ICCV2019 #ComputerVision #DeepLearning.
2
8
50
I keep reading in papers that RANSAC is so excruciatingly slow that getting rid of it is a contribution. Watch DSAC* estimating poses from thousands of correspondences in 30ms per frame, including network execution and RANSAC. 👇. Brrr.
3
3
50
Who said re-localisation in changing conditions is a problem?.
4
0
49
Only 6650 actual submissions to #CVPR2020. End of the world prevented. (3350+ people were just curious what the current submission ID might be?).
1
17
49
@eccvconf Oh, this is just cruel. Posting this and then it's still that 14 page, 1 column "aesthetic".
3
0
48
There is this magical moment of excitement and joy, right after drafting a new approach, and before actually implementing it and seeing it fail in all the experiments.
2
3
46
Extending on my earlier post about research trends at #CVPR2018, I wrote a little python script that plots topic popularity (measured by key word matches against paper titles) over time. It's a simple Jupyter notebook, so you can play around yourself:
1
24
45
Hopefully, someday I will review a paper that starts with: . "X is not a fundamental problem in computer vision. But we looked into it anyway.".
3
0
47
Thanks @ducha_aiki for sharing our work faster than we could get it to arXiv. Its available there as well, now. 📜 TL;DR? Upload the gist of it into your brain via YouTube: .▶️ w/ @axelbarrosotw, @viprad, Gabe, and @dantkz
Two-view Geometry Scoring Without Correspondences. Axel Barroso-Laguna, @eric_brachmann Victor Adrian Prisacariu Gabriel Brostow @dantkz . tl;dr: in title + nice analysis of the metrics for the epipolar geometry.I almost want to write a blogpost-review :).
2
14
46
Not long until the 9th(!) Workshop on Recovering 6D Object Pose (R6D) at #ECCV2024, Sunday AM. Great speakers, and @vannguyen_ng, @tomhodan and @ma_sundermeyer will tell us about the #BOP Challenge 24 - the challenge is still running, but you get to see early bird results!
4
6
47
How to explain the difficulties of hyper-parameter tuning to non-experts.
2
1
42
Naver Labs is joining the #MapFreeReloc challenge as co-sponsor, giving $2000! . Thanks @JeromeRevaud and @naverlabseurope for making it happen! . We have a total of $6000 in prizes for the challenge winners.
0
4
43
I really like that I'm not able to rate "borderline" in the final #ICCV2021 assessment. I'm split regarding several submissions, now I have to think twice as I have to lean in some direction. An unexpectedly enjoyable inner fight.
0
2
44
Turning regression into classification problems is one of the most unappealing workarounds we currently often resort to, I find.
5
3
44
6/6 #CVPR2022 reviews done. Strong stack this one: worst score is borderline - never had this before. Innovative ideas and above-average writing. I'm impressed.
0
0
44