After 2 years of hard work by the team, we are thrilled to release today! Scholar Inbox is a personal paper recommender which enables you to stay up-to-date with the most relevant progress by delivering personal suggestions directly to your inbox.🧵
Why are Tesla
@elonmusk
and Wayve
@alexgkendall
@Jamie_Shotton
moving towards end-to-end autonomous driving? What is the state-of-the-art in this field? With our friends
@francislee2020
we recently wrote an extensive survey paper on this emerging topic:
🎅 Today, we have an early Christmas present for you: SDF Studio. Building on the fantastic
@nerfstudioteam
code, we have integrated various implicit surface reconstruction techniques in one common framework!
More algorithms and results coming soon..
This is unbelievable: Michael Niemeyer
@Mi_Niemeyer
won the CVPR 2021 best paper award!! Thanks to Michael and all my fantastic students and postdocs for their extraordinary work! You are just amazing..
We have now integrated Neuroangelo into SDF Studio. Works like a charm! Get it from here and - if you are at
@CVPR
- visit our demo today where you can see SDF Studio in action!
I am hiring PhD students and PostDocs! If you like to join a great team to conduct curiosity-driven research on implicit neural 3D representations join us now! Flyer:
Despite its relevance, it can be hard to get into RL. This motivated us to write "An Invitation to Deep Reinforcement Learning", a tutorial for readers with only basic ML knowledge. Huge kudos to
@bern_jaeger
for this effort. Help us to spread the word!
Just finished recording my last lecture of our self-driving cars course this winter. The entire course including videos, slides and problem sets is now available online!
Did you know that SotA flow, stereo and depth results can be obtained using simple winner-takes all feature correlation without cost-volume post-processing? Impossible? Then try out our new UniMatch model:
To level student's background knowledge for our deep learning lecture, I recorded a series of micro tutorials on relevant concepts (linear algebra, differential calculus, probability/information theory).
Slides:
Videos:
Geometric Transform Attention (GTA) is a principled way to encode 3D geometric structure of tokens into vision transformers. This work was done by
@takeru_miyato
in collaboration with
@wellingmax
and
@bern_jaeger
. Project page & code:
In RegNeRF we investigate how geometry and appearance can be regularized through rendered patches from unobserved viewpoints. This enables training Mip-NeRF from only 3 images. Great collaboration with the Google team.
Little reminder about our self-driving cars lecture with 1000+ slides and >20h video lectures which we made public. Pen and paper as well as coding exercises will follow during the semester!
We have an opening for a PhD student or PostDoc to build the next-generation of generative 3D models! This will be a collaboration with
@fedassa
@Mi_Niemeyer
@mjoech
from Google Zürich. We are looking forward to you joining our team! Flyer:
What is the best representation for radiance fields? We consider a feature volume as a 4D tensor and propose to factorize this tensor into multiple compact low-rank components leading to state-of-the-art results in <10 min using pure PyTorch.
@eccvconf
.
We received this year's teaching award for our lecture Computer Vision! All slides, videos and problems are available online. Hope they are useful also to people outside Tübingen!
3D Gaussian Splatting is fantastic, but suffers from severe aliasing when changing the focal length or depth from the one used during training. Mip-Splatting addresses this problem in a principled way and provides alias-free reconstructions and renderings.
Depth and normal cues extracted from monocular images are complementary to reconstruction cues. We show that they significantly improve the performance of implicit surface reconstruction methods. New work by Zehao Yu et al. to appear at NeurIPS 2022:
One of our most surprising findings this year: Perceptual features in the discriminator dramatically improve GAN convergence and FID scores across SotA models, but it only works when adding deterministic random projections! We still don't know exactly why.
Super happy for my brilliant PhD student Zehao Yu for winning the CVPR best student paper award with his Mip-Splatting idea! Joint work with
@AnpeiC
and
@SattlerTorsten
as part of
@ELLISforEurope
.
KITTI has received the ICCV 2021 Everingham prize! Big thanks to all collaborators and the award committee for this recognition! With Philip Lenz, Christoph Stiller and
@RaquelUrtasun
.
Excited to announce our new work on a unified formula for neural fields (Factor Fields) and a novel dictionary factorization (Dictionary Fields, SIGGRAPH 2023)!
Wohoo! Congratulations to Michael Niemeyer
@Mi_Niemeyer
for his GIRAFFE work being selected as a CVPR'21 best paper candidate. 32 out of ~10k submissions have been nominated, amongst those 4 papers from Tübingen! 🎉🍾🥳
The Autonomous Vision Group offers research fellowships for Ukrainian PhDs/PostDocs/researchers at risk in CV/NLP/ML as well as software engineering positions for BSc/MSc students. Please contact us for details and forward this to anyone this might apply to. Thanks.
UniMatch, our unified framework for flow, stereo and depth estimation has been accepted by TPAMI! Congratulations to
@haofeixu
and team for this fantastic work, demonstrating how far strong features can take us, even with simple winner-takes-all matching.
Does training of generalizable 3D vision models have to be slow? In LaRa, we train a feed-forward 2DGS model in only two days using just 4 GPUs on Objaverse. Great work by
@AnpeiC
in collaboration with
@haofeixu
and
@SiyuTang3
. Paper and code:
ELLIS PhD program 2024 call for applications: Apply and join one the largest PhD and PostDoc program on machine learning and AI in Europe! Deadline: November 15.
Can we use explicit representations to model implicit surfaces? In Shape as Points we propose a differentiable Poisson solver which does exactly that. Lightweight. Interpretable. 10x faster than neural implicits (eg, Occupancy Networks).
ATISS is a new model for indoor scene synthesis which generates a room based on its floor plan. We consider rooms as unordered sets of objects and train our model to be permutation invariant which is critical when working with sets.
We will be presenting 2 papers at SIGGRAPH in 2 weeks: 2D Gaussian Splatting and Binary Occupancy Fields. Both tackle the problem of accurate surface reconstruction using different techniques.
Excited to announce VoxGRAF w/
@K_S_Schwarz
@AxSauer
@Mi_Niemeyer
: Motivated by INGP, we bring good old voxels back to the game of 3D aware image synthesis! Cool result: Rendering from any view is super fast (5 ms) after generating a feature grid (200 ms).
ICCV is nearing its end. I compiled a little playlist with talks I gave this year, covering datasets, metrics, neural scene representations, human avatars and self-driving. I hope we see each other live again next year!
Today's GANs suffer from high frequencies artifacts. While most works attribute these to the generator, other works point to the discriminator. We rigorously analyze both the generator and discriminator in toy settings to shed light onto this problem.
In GraphDreamer we distill editable compositional 3D reconstructions from 2D diffusion models using scene graphs as intermediate representation. To appear at CVPR 2024. Paper & Code:
The Scholar Inbox team and ambassadors are ready for
@CVPR
! All attendees will receive an email within the next days to create their personal CVPR program.
Crazy AF. Paper studies
@_akhaliq
and
@arankomatsuzaki
paper tweets and finds those papers get 2-3x higher citation counts than control.
They are now influencers 😄 Whether you like it or not, the TikTokification of academia is here!
Did you ever want to create a 3D avatar of yourself using only an RGB video from your smartphone? Now you can with ARAH: Animatable Volume Rendering of Articulated Human SDFs. Joint work with Shaofei
@SiyuTang3
@K_S_Schwarz
to appear at
@eccvconf
:
So happy about our first NLP paper accepted to
@COLM_conf
! A new transformer attention kernel that incorporates document structure. Increases efficiency and stability, and leads to hierarchical representations. Great collaboration with TU Darmstadt!
We made SNARF 150x faster!! Fast-SNARF enables very fast and simultaneous optimization of shape and skinning weights given deformed observations without correspondences. Joint work with
@XuChen71058062
@OHilliges
@Michael_J_Black
. Code available at:
Monocular geometric cues do not only help reconstruction but also open world object detection! Check out our new ICLR'23 paper GOOD: Exploring geometric cues for detecting objects in an open world:
Joint work with
@hhwpku
@isDanZhang
The Autonomous Vision Group is looking for a PhD student at the intersection of NLP and CV to help revolutionize scientific discovery in the age of exponential research growth. Please help share with potential candidates!
Happy to announce that we have a new PostDoc! He is called Markus and will work on 3D Gaussian Splatting for animal reconstruction. He is very calm and approachable. Everybody loves him.
Glad to find out this morning that our Survey on End-to-End Autonomous Driving has been accepted to TPAMI! This work has been a collaboration between
@francislee2020
's team and ours (
@kashyap7x
,
@bern_jaeger
). Paper and Links:
Excited to share that just 1.5 years after submission, Carolin's TPAMI paper finally got accepted! This was a big project with many ups and downs. Finally, we were able to show some pretty nice material reconstructions with our self-made hand-held scanner:
Since 3 months (silence period) have passed, I am now happy to share the talk I gave in the CVPR'23 workshop on Common Misconceptions in Autonomous Driving: The respective papers appear at ICCV'23 and CoRL'23.
I am super excited to announce that in 3 weeks at ICCV we will be presenting AG3D, a model that learns to generate novel realistic 3D humans from 2D images alone. It wasn't clear at all if this could work. The more exciting we got about the results:
Did you know? has been created by scientists for scientists. As we are using it ourselves, we understand the community's needs. Of course, we also appreciate your feedback! Together, let's give scientists the best tools so they can focus on science!
Did you know? doesn't just recommend articles that are trending on social media. also recommends articles that don't get as much attention but are equally relevant to your research interests. Sign up today to look beyond the hype!
We at love open access and believe that all research should be freely available! We therefore index all 2.4M arXiv articles, covering physics, math, computer science, quant. biology, finance, statistics, electrical engineering and economics.
Attending
@CVPR
next week and wondering which of the 2719 posters to visit? Register now at to find out! Scholar Inbox is a community-driven paper recommender system developed by researchers for researchers.
Today, I assumed my 3 year term as head of the department of computer science at
@uni_tue
. First task completed: Organized our faculty retreat at castle Weitenburg!
We are looking for a motivated postdoctoral researcher interested in NLP for scientific document analysis! If you know someone interested - let us know. Flyer:
Given a monocular video, IntrinsicAvatar learns animatable clothed human avatars with decomposed intrinsic properties including albedo, material, and geometry. It was a great
@ELLISforEurope
collaboration with
@SiyuTang3
,
@sfwang0928
and
@anticboz
!
Our Deep Learning lecture at
@uni_tue
is featured by Analytics India Magazine as one of the top 15 YouTube channels to follow for deep learning enthusiasts!
🎉 Proud and excited to announce that our team won the 2023 nuPlan Challenge! This is the first large-scale benchmark for real-world vehicle motion planning. Kudos to
@DanielDauner
who was the primary contributor. Team:
@MHallgarten0797
,
@AutoVisionGroup
, and me.
ATISS is
@paschalidoud_1
's new work on generating, completing and manipulating indoor scenes at the object level. It's fast and leads to consistent results. Nice collaboration with
@FidlerSanja
's team.
We are super excited to release our latest generative model StyleGAN-XL which will appear at SIGGRAPH 2022: . StyleGAN-XL achieves SotA results on ImageNet and FFHQ 1K, even outperforming recent diffusion models. Great work by
@ax_sauer
and
@K_S_Schwarz
!🧵
We finally released all benchmarks for the KITTI-360 dataset and got our TPAMI paper on KITTI-360 accepted! Huge shout out to Yiyi Liao for her hard work on this. Is KITTI-360 relevant to you? Find out in her new blog post:
The Tübingen AI Center is looking for 2 postdoctoral researchers interested in computer vision and machine learning! Flyer:
Feel free to forward this flyer to suitable candidates. We are looking forward to working with you on exciting research problems!
MetaAvatar is a practical approach to generate realistic animatable 3D human avatars from as little as 8 depth maps (eg. Kinect) in only 2 minutes. Key to this is a meta-learned hypernetwork and a model of pose-dependent deformations in canoncial space.
🚙Excited to release CARLA Garage (a set of SOTA self-driving baselines for CARLA) alongside an
#ICCV2023
paper which identifies hidden biases and uncovers the secrets of self-driving on CARLA:
.
Joint work with
@Kait0o0
@kashyap7x
.
🚨Paper Alert 🚨
➡️Paper Title: HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting
🌟Few pointers from the paper
🎯Holistic understanding of urban scenes based on RGB images is a challenging yet important problem. It encompasses understanding both the geometry
Want to do your ML/CV/NLP/Robotics PhD with two top researchers in Europe? Apply to the ELLIS PhD program, the central entry point for excellent ML research in Europe!
ELLIS PhD Program in
#machinelearning
now accepting applications (deadline: November 15, 2021). Details on the program and the application process here:
We are excited to provide personalized conference programs for
@iclr_conf
next week! If you like to access your personal conference program based on your research interests, register at !
@haofeixu
's MuRF is a new feed-forward NVS model with SotA performance on sparse small and sparse large baseline settings. Key for efficiency and quality: a convolutional radiance decoder in the target frustum. Code:
PlanT (CoRL'22) is a learned planner for self-driving based on object-level representations and a transformer architecture which can explain its decisions by identifying relevant objects. Joint work with
@KatrinRenz
@kashyap7x
@MerceaOtniel
@zeynepakata
The preprint about our new KITTI-360 dataset and its benchmarks and challenges is now online! We hope many people will find this dataset useful and that it will push self-driving and research at the intersection of vision, robotics and learning.
IntrinsicAvatar: Given a monocular video, IntrinsicAvatar learns animatable clothed human avatars with decomposed intrinsic properties including albedo, material, and geometry. Project and Code:
We thank VolkswagenStiftung for a Momentum research grant to support our project which aims to accelerate research through AI based research recommendations and analysis. Our first mission is to support
@CVPR
2023 with personalized conference programs.