3D Gaussian is great, but how can you interact with it 🌹👋? Introducing #PhysDreamer: Create your own realistic interactive 3D assets from only static images! Discover how we do this below👇 🧵1/: . Website:

Attending CVPR at Seattle this week. Happy to chat about anything!

Got a chance to play pingpong in VR with this virtual agent on May, it’s so cool! .Imagine more sophisticated interactions with virtual agent in the future.

Impressive reconstruction(scene and object) results. This gives me a feeling that attention is all you need for 3D reconstruction.

Really impressive results. I think data driven approaches will be able to do fully inverse/forward rendering soon, including strong specular effects, hard shadows and transparencies.

@zhu_zhaocheng Feels like the biggest problem is that JAX has smaller opensource communties compared to torch on most area now.

Excited to share our new work on Autonomous driving:.@yuewang314, Vitor Guizilini, Yilun Wang, @zhaohang0124, @JustinMSolomon . DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries . Our work allows end-to-end multi-camera 3D detection.

@taiyasaki @vincesitzmann Very Good demos. But feels like, to get better results with fewer images, LFN needs more modeling on the scene, thus drops one big advantages over NeRF: no explicit modelling/contraints of the rendering process. -- Using depth reg assumes near Lambertian scene and no occlusion.

First dinner at Berkeley!&,@;);/?

🎥 We represent 3D objects as 3D Gaussians and synthesize a 2D video of the object in motion. We estimate the materials through differentiable simulation and differentiable rendering. Check out more results at our project page: 4/N

So cool.

@Xianbao_QIAN Real world model should be much more complex and capable than this.

Realistic interaction requires physical materials of the 3D objects, yet these materials can be spatially varying and are hard to estimate from static images 🥲. However, video generation models, having seen millions of videos 🎬, contain visual priors of object dynamics. 2/N

@ShivamDuggal4 @MITEECS @MIT_CSAIL @pathak2206 @CarnegieMellon @roboVisionCMU Cograts!.

@peterljq @xuyilun2 @CongyueD @ishaanpreetam @TianweiY @ShivamDuggal4 @_atewari 🤣🤣.

Interesting findings with tons of experiments!.

@james_y_zou @mertyuksekgonul @federicobianchy @ria_kalluri @jurafsky Exciting work! I feel similar problem also occurs on stable-diffusion. Where generated images hardly follow the composition of the text prompt.

@jbhuang0604 @CVPR @overleaf And just now it broke Cmt.

Work done with @Koven_Yu, @ChrisWu6080, Brandon Y. Feng, Changxi Zheng, @Jimantha, @jiajunwu_cs, and Bill Freeman. 5/5.

Soon there gona be a list of Tiny /Quabntized . BERT papers.

@Aaronf_hd @janusch_patas I also use metashapes and it’s way faster and robust. But i don’t know why. One general assumption is that heavy engineering is very important for implementing algorithm with long pipeline like SFM, and metashapes just have the money to engineering the system quite well.

@UUUUUsher Thanks!.

@dr_cintas So cool! From these demo videos, the speed of generative feels to be “real time”(few seconds). That’s quite impressive if they are using diffusion models on a cpu machine.

@liangchensong @taiyasaki @vincesitzmann Thanks for the replying. I think your work is a good one, it shows a good balance between more prior and being versatile. Looking forward to your future works!.

@tolga_birdal @CVPR Agree！.

@timudk @rbhuta95 Thanks for the sharing! Is there a way to control camera motion magnitude and scene motion magnitude separately?.

@Bw_Li1024 @eccvconf @DrChenWang @pranay_ar @seungchankim25 @smash0190 Congrats!.

@TairanHe99 Congrats!.

@Xianbao_QIAN Good question! It's slow. We mentioned the speed in the final Limitation section of the paper. Our implementation takes 1 min of a V100 GPU to produce 1 second of the simulated video.

@rrika9 remind me of the range analysis paper on sdf: Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysis.

We introduce #PhysDreamer. The key idea is to distill the object dynamics priors learned by a video generation model to estimate the physical materials of static 3D objects. 3/N