Boyi Li
@Boyiliee
Followers
2K
Following
468
Media
24
Statuses
114
Joined March 2020
Iโve dreamt of creating a tool that could animate anyone with any motion from just ONE imageโฆ and now itโs a reality!.๐ Super excited to introduce updated 3DHM: Synthesizing Moving People with 3D Control. ๐บ๐3DHM can generate human videos from a single real or synthetic human
4
38
192
๐ Introducing ๐๐จ๐ฅ๐ ๐บ: a mixture-of-experts video captioning framework that outperforms GPT-4V and Gemini-Pro-1.5 in general scenes ๐ผ๏ธ, autonomous driving ๐, and robotics videos ๐ค. ๐:
8
63
201
Super excited to announce our new work: Synthesizing Moving People with 3D Control (3DHM)๐ก. Why is 3DHM unique?.With 3D Control, 3DHM can animate a ๐ฟ๐ฎ๐ป๐ฑ๐ผ๐บ human photo with ๐ฎ๐ป๐ poses in a ๐ฏ๐ฒ๐ฌ-๐ฑ๐ฒ๐ด๐ฟ๐ฒ๐ฒ camera view and ๐ฎ๐ป๐ camera azimuths from ๐ฎ๐ป๐ video!
15
53
241
Can we ask a robot to make boba milk ๐ง?.Can we ask a robot to make any drinks based on limited task guidelines? ๐.Can we change our minds and interact with the robot during the execution? ๐ค. Yes for all!!. We are excited to announce ITP (Interactive Task Planning with Language
5
47
213
Semantic segmentation is not limited to a fixed label set! Happy introduce LSeg, a novel model that can dynamically handle arbitrary label sets on the fly with varying length, content, and order ๐ป.Paper: Demo:
4
24
112
๐ค Our "Vision and Language for Autonomous Driving and Robotics" full-day workshop @CVPR will take place next Tuesday. Please check the details here: See you in Seattle!
We are happy to announce the first Vision and Language for Autonomous Driving and Robotics (VLADR) workshop at @CVPR 2024! . Call for contributions and more details ๐๐ป. See you in Seattle! ๐.
1
19
91
@brjathu @YGandelsman @JitendraMalikCV @geopavlakos @goelshbhm @jane_h_wu Paper on Hugging Face: Website:
0
3
26
WiCV 2021 @CVPR is coming soon (June 19th), please feel free to share this message or send the application link to anyone who needs help๐ณ.
With the generous funding received from our sponsors, we are happy to offer extra spots for WiCV workshop in the upcoming CVPR. Please fill out this form latest by Jun 18th to be considered for the conference registration funding:
0
2
15
Can we ask a robot to make boba milk ๐ง?.Can we ask a robot to make any drinks based on limited task guidelines? ๐.Can we change our minds and interact with the robot during the execution? ๐ค. Yes for all!!. We are excited to announce ITP (Interactive Task Planning with Language
1
0
12
@NVIDIAAI @nvidia Proudly working with an incredible team @yuewang314, @PointsCoder, @iamborisi, @Veer_Sushant, Karen Leung and @drmapavone @NVIDIAAI @nvidia #GTC2024 โณ๏ธ. ๐arxiv: ๐ปproject page: ๐related link: @_akhaliq.
Nvidia presents Driving Everywhere with Large Language Model Policy Adaptation. paper page: Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of
0
0
8
@CVPR Glad to share our LLaDA demo! @CVPR. working with @yuewang314,@PointsCoder,@iamborisi,@Veer_Sushant, Karen Leung and @drmapavone @NVIDIAAI @NVIDIADRIVE.
0
0
8
We will host keynote talks by @trevordarrell, @JitendraMalikCV, @chelseabfinn, @xf1280, and Long Chen, as well as oral & poster sessions!. Co-organizing the workshop with @yuewang314, @zhaohang0124, @JiaweiYang118, @PointsCoder, @SergeBelongie, @FidlerSanja, and @drmapavone!.
0
0
8
@tsunghan_wu @LongTonyLian @profjoeyg @trevordarrell @berkeley_ai @CVPR - code: - arxiv: - project page:
0
0
6
Video understanding is challenging, and currently, a single VLM cannot capture everything in a video. Wolf aims to produce accurate descriptions of scenes, agent motion, and videos by introducing a novel framework, the CapScore metric, and four human-annotated datasets.
1
0
9
Super honored to work with @brjathu @YGandelsman Alexei A. Efros, and @JitendraMalikCV!. We also thank the insightful discussions with @geopavlakos @goelshbhm , and @jane_h_wu!.
1
0
6
3DHM training pipeline is self-supervised and scalable, it can be trained with any human videos. Please visit our website for more information:
1
1
6
Joint work with @KilianQW, @SergeBelongie, Vladlen Koltun and @ranftlr1. @cs_cornell @cornell_tech. @uni_copenhagen, @Apple, @IntelAI.Code is available at Any feedback is welcome and appreciated.
1
0
5
๐ก Motions from Text.Text Input: A person turns to his right and paces back and forth.
1
0
4
To evaluate caption quality, we introduce CapScore, an LLM-based metric that assesses the quality and similarity of generated captions to ground-truth captions. We built 4 human-annotated datasets in 3 domains: AV, robotics, and general scenes, to enable comprehensive comparisons.
1
0
3
Wolf achieves superior captioning performance compared to SOTA methods from the research community (VILA1.5, CogAgent) and commercial solutions (Gemini-Pro-1.5, GPT-4V). For example, Wolf improves caption quality by 55.6% over GPT-4V on challenging driving videos (by CapScore).
1
0
2
0
0
3
@LigengZhu @tianran_ @stan188249301 @Yuxiao_Chen_ @Yao__Lu @YinCuiCV @Veer_Sushant @DrQueuecumber @PhilionJonah @xinshuoweng @XueFz @DrJimFan @yukez @jankautz @liu_mingyu @FidlerSanja @iamborisi @trevordarrell @JitendraMalikCV @songhan_mit @drmapavone @NVIDIAAI @nvidia @berkeley_ai @MIT_CSAIL @StanfordAILab @UTCompSci (Also thanks @_akhaliq for featuring our work!).
Wolf. Captioning Everything with a World Summarization Framework. We propose Wolf, a WOrLd summarization Framework for accurate video captioning. Wolf is an automated captioning framework that adopts a mixture-of-experts approach, leveraging complementary strengths of
0
0
3
@ceyda_cinarel Very happy you love them ๐โค๏ธ! These are largely inspired by the discussion with @SergeBelongie ,@YinCui1 and @TsungYiLin1.
0
0
3
๐ ๏ธ Explore the SLD pipeline: a simple yet effective โtraining-freeโ approach that combines LLM-integrated detectors for image assessment with base diffusion models for actual image modification.
1
0
3
@nmervegurel @jieyuzhao11 @mariyaivasileva @linh_trd @MeeraDesai18 @siri_r @AmakaOraekwe @akikoe_ @NanaYaaSally @geeticka1 @S_A_Lee_ Aga here๐ท๐.
0
0
3
Proudly working with an incredible team @tsunghan_wu, @LongTonyLian, @profjoeyg, and @trevordarrell at @berkeley_ai @CVPR #CVPR2024. Excited for this journey in exploring LLM in content creation! ๐.
1
0
2
@CVPR Co-organizing the workshop with @yuewang314 @zhaohang0124 @JiaweiYang118 @PointsCoder @SergeBelongie @FidlerSanja and @drmapavone!.
0
1
3
๐๐ ๐ข๐ง๐ฏ๐ข๐ญ๐ ๐ญ๐ก๐ ๐๐จ๐ฆ๐ฆ๐ฎ๐ง๐ข๐ญ๐ฒ ๐ญ๐จ ๐ฉ๐๐ซ๐ญ๐ข๐๐ข๐ฉ๐๐ญ๐ ๐ข๐ง ๐ญ๐ก๐ ๐๐จ๐ฅ๐ ๐๐ก๐๐ฅ๐ฅ๐๐ง๐ ๐: ๐๐ง๐ ๐ฎ๐ฌ๐ ๐๐จ๐ฅ๐ ๐ข๐ง ๐ญ๐ก๐๐ข๐ซ ๐๐๐ฉ๐ญ๐ข๐จ๐ง ๐ฉ๐ข๐ฉ๐๐ฅ๐ข๐ง๐!.
1
0
2
In conclusion, Wolf can (1) generate high-quality captions for challenging autonomous driving videos;.(2) provide a general benchmark for video caption qualities.
1
0
2
Further, Wolf can generate high-quality captions to improve multimodal foundation models. VILA-1.5, when fine-tuned with Wolf-provided captions on 500 highly interactive nuScenes videos, significantly outperforms the original VILA-1.5 model.
1
0
2
@LigengZhu @tianran_ @stan188249301 @Yuxiao_Chen_ @Yao__Lu @YinCuiCV @Veer_Sushant @DrQueuecumber @PhilionJonah @xinshuoweng @XueFz @DrJimFan @yukez Advisory Board: @jankautz, Andrew Tao, @liu_mingyu, @FidlerSanja, @iamborisi, @trevordarrell, @JitendraMalikCV, @songhan_mit, @drmapavone!.@NVIDIAAI @nvidia @berkeley_ai @MIT_CSAIL @StanfordAILab @UTCompSci.
1
0
2
@Karttikeya_m @gberta227 @cfeichtenhofer @chaoyuaw @tyleryzhu @JaiswalNitya96 @_amirbar Great work!!.
0
0
2
@ylzou_Zack Thanks @ylzou_Zack! I really like this question, as has been mentioned in the paper, we assume due to memory constraints, the image encoder predicts pixel embeddings at a lower resolution than the input image resolution.
1
0
1
@Haz09714143 Thanks for your question and interest in LSeg! We've updated detailed instructions in our code: Please let us know if you have any questions! Any feedback is welcome and appreciated.
0
0
0
@brjathu @YGandelsman @JitendraMalikCV @geopavlakos @goelshbhm @jane_h_wu We hope 3DHM could contribute valuable insights and encourage further advancements in research, with an emphasis on the alignment of pixels with 3D and the integration of video generation with 3D control. ๐ก.
1
0
1
Proudly working with an incredible team: @LigengZhu, @tianran_, @stan188249301, @Yuxiao_Chen_, @Yao__Lu, @YinCuiCV, @Veer_Sushant, @DrQueuecumber, @PhilionJonah, @xinshuoweng, @XueFz, @DrJimFan, @yukez!.
1
0
1