Jia-Bin Huang
@jbhuang0604
Followers
59K
Following
8K
Media
2K
Statuses
5K
Associate Professor @umdcs; I like pixels.
College Park, MD
Joined May 2015
How can we reconstruct 3D objects under *extreme lighting variations*? π€οΈπ₯οΈπππ. How about . π€ appearance embedding? .BUT it cannot capture view-dependent appearances. π€ inverse rendering? .BUT it suffers from ambiguities. π‘ Our idea: Relighting comes to the rescue!
3
17
126
How to start a presentation?. Most people start with all the boring info (e.g., repeating everything your host just said) and unmotivated technical details. π₯±. After a few mins, everyone starts checking their phone! π±. How to avoid this? Here is a simple template. π§΅.
70
1K
6K
Super excited to share our work on video completion at #ECCV2020!π€©Our method seamlessly removes objects, watermarks, or expands field-of-view from casually captured videos. Paper: Project: With @gaochen315, Ayush, and @JPKopf
68
916
4K
How to work with your advisor(s)?. Working effectively with your advisor is the no doubt the key to success of your research! However, junior grad students often don't have a clear idea on how to do so. Sharing some tips that I found useful. π.
31
740
3K
How to make steady progress in my research?. I worked so damn hard but "IT JUST DOESN'T WORK!"π€ .How can I unblock myself quickly and make good progress toward the goals? . Below I compiled a list of tips that I found useful. π.
17
546
2K
How to come up with research ideas?. Excited about starting doing research but have no clue?π€·ββοΈπ€·π»ββοΈ Here are some simple methods that I found useful in identifying initial directions. Check out the thread below π.
39
522
2K
How to write a paper that looks like a good one?. You worked super hard and did great research, but somehow the reviewer 2 just doesn't buy it. Why? π€. It's probably because your paper does not look like a good paper *visually*. π. How? πππ #AcademicTwitter.
24
458
2K
Research summary for the last 3 years. 2021: Replace every CNN with a Transformer .2022: Replace every GAN with diffusion models.2023: Replace every NeRF with Gaussian splatting.
33
239
2K
Reviewer: Weak Reject!. This is a straightforward combination of two well-known methods. The novelty is limited.
24
137
2K
R2: While the results are impressive, this is a simple combination of diffusion transformer (ICCV 2023) and latent diffusion model (CVPR 2022). Limited novelty. Weak reject.
Introducing Sora, our text-to-video model. Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. Prompt: βBeautiful, snowy
55
151
2K
Holly crap! . Itβs so embarrassing that I have a PhD and have been actively working in AI but my presentation does not even come close to 1/10 of the clarity of this video.
The next chapter about transformers is up on YouTube, digging into the attention mechanism: The model works with vectors representing tokens (think words), and this is the mechanism that allows those vectors to take in meaning from context.
17
116
2K
How to network at a conference?. Many students will be attending their first-ever *in-person* conference this year. How exciting! π€©. Some tips on making the best of attending a conference. π§΅.
22
308
1K
How to do research with my mentors effectively? . I get this question frequently in my open office hours. I am still learning as well but I hope sharing my βπ° may be helpful to some. Key idea β‘οΈ **Help them help you!**. How? Check out the thread π§΅.
18
385
1K
How to write the Introduction?. As a junior student, writing the introduction of a research paper is arguably the most daunting part of paper writing. π±. Here is a simple template I find useful: . 3 Figures πΌοΈ + 5 Questions π€.
17
290
1K
I always feel frustrated at taking photos through fences or windows (e.g., when taking my kids to the zoo). Check out our #CVPR2020 work on "learning to see through the obstructions". Paper: Code: Colab:
13
352
1K
How to keep track with the literature?. Getting started with research but overwhelmed by THOUSANDS of papers each year? How could one stay sane keeping track of the literature? π±. Here are some tricks I found useful. π§΅.
19
320
1K
How to write clear and concise sentences?. Getting ready to write up your very first research paper? Writing a paper could be daunting, particularly for non-native English speakers. π¬π¬π¬ How can we avoid common mistakes in technical writing?. Check out the thread below! π§΅.
13
301
1K
Introducing Generative Omnimatte:. A method for decomposing a video into complete layers, including objects and their associated effects (e.g., shadows, reflections). It enables many cool applications, such as video stylization, compositions, moment retiming, and object removal.
23
173
1K
Sharing one idea I found useful for paper writing:. Do NOT ask people to solve correspondence problems. Some Dos and Don'ts examples below:. *Figures*: Don't ask people to match (a), (b), (c) . with the descriptions in the figure caption.
9
305
1K
Annoyed by shaky videos? Check out our work on stabilizing shaky handheld videos *without cropping*!. Neural Re-rendering for Full-frame Video Stabilization. Project:
7
157
918
How to end a presentation?. You have kicked off your talk with a fascinating story and presented an amazing body of work. π€©. So now how do you end your talk? π€. Check out a simple template I like! π§΅.
6
165
902
Somehow my timeline is filled with papers where the models were trained with 64 A100 GPUs for weeks. Grad students with two 2080Ti:
23
74
884
Check out our #SIGGRAPH2020 paper on Consistent Video Depth Estimation. Our geometrically consistent depth enables cool video effects to a whole new level!. Video: Paper: Project page:
10
208
863
As my kids are singing APT non-stop these days, I did a bit of reverse engineering of the APT music video and tried to understand why the MV is so addictive. Here is what I learned.
25
85
892
Most salient progress in AI/ML these days are driven by insanely large amount of data, model, and compute. It seems increasingly hard to do research in this space in an academic setting. π. What should poor academics do next? .(asking for a friend).
83
73
824
How to intern? . Research internships are great opportunities to do cool research in industrial labs! But it's definitely not easy to complete a solid work in 3 months. Some tips for having a successful internship . π§΅.
10
155
842
Sharing tips on preparing your presentation slides. Just attend many thesis presentations and qual exams at the end of the semester. I compiled some common pitfalls here and hopefully would be helpful to some. Check out the thread π§΅below!.
12
208
826
Have you ever wondered why papers from top universities/research labs often appear in the top few positions in the daily email and web announcements from arXiv?. Why is that the case? Why should I care?
21
154
817
PhD then: .βWe validate our method on a large-scale dataset of hundreds of images.β. PhD now: .βWe validate our method on 10 different modalities, 15 domains, 20 scenarios, 25 tasks, and 200 languages, each with millions of testing examplesβ.
15
51
802
Based on the twitter timeline, only the top 10 universities are accepting new PhD students.
16
26
757
How to communicate clearly?. Majority of our time goes into emails, slack messages, and loads of meetings. Yet, as a student, no one taught me how to communicate well. (I must have annoyed a lot of people along the way . π¬). Here is a template for organizing your message.
11
124
757
Writing Related Work. I enjoy reading/writing the related work section of a paper. It helps organize prior research and put the contributions of the work in proper context. But HOW? Check the thread belowπ.
6
171
750
How to find research opportunities?. Finding opportunities to gain experience is arguably the most challenging part for students wishing to pursue grad school, particularly for those who don't have resources/connections. Some tips on approaching potential mentors. π§΅
20
141
754
Why is self-supervision in vision still not working? π€. When pretraining a transformer on TEXT-only data by predicting the next tokens, we see clear improvement trends as we scale the model, data, and computing. But after trying to pretrain a transformer on IMAGES-only data.
73
48
739
Sharing ideas on how to disseminate your research. "I am THRILLED to share that our paper is accepted to . ". Congrats! So what's next? No one is going to browse through the list of thousands of accepted papers. Ain't nobody got time for that. Check out π§΅below for examples.
11
177
716
For the past month, life slowed down in ways I never expected. I thought I was healthy until I wasnβt. But itβs in these moments I gain clarity and perspective. Now, onto recovery and looking forward to getting back to things I love soon!
124
5
714
What's the differences among . Latent space, feature space, embedding space, representation space, latent feature, feature embedding, latent representation, embedding representation, latent embedding, and feature representation? π€.
21
68
696
Me to students: "Writing a research paper sharpens your mind, trains you to think critically, and helps you communicate effectively. These skills during your PhD training are highly transferable.". Also me: \vspace{-2.37mm}.
9
35
712
How to Survive the First Year of PhD?. Excited to get started your PhD study this Fall? π€©.How to make the best out of your first year and set yourself up on the right path?. Here are a few tips I wish I knew. π§΅.
14
138
697
How to get unstuck?. "It doesn't work." π€·ββοΈπ€·ββοΈ. In most research projects, ~99% of the time your experiments DO NOT work. What should we do to get ourselves unstuck? π€. Sharing some tips I found useful. π§΅.
5
132
672
The more I learn vision-language models, the more I realize how little *vision* there is in them. π€. Model: Image as a sequence of tokens (+connectors).Pre- and post-training: Natural language supervision. No feature detection, no grouping, no objects, no shape, no 3D.
22
51
686
college admission in a few years.
NeurIPS 2024 will have a track for papers from high schoolers.
7
60
682
How to choose a good Ph.D. advisor?. Grad school application season is coming to an end. Congrats on the offers! Now, how would you choose your advisor that will help you succeed? . Some tips below (that I wish I knew).
4
108
664
How to meet with your advisors/mentors?. If you are a grad student, having effective regular meetings with advisors or mentors is absolutely crucial for your success! . Here are some tips on how to make the most of it! π§΅
7
106
651
A paper's impact on your career. A good reminder from Bill Freeman.
17
59
653
Stats of students approaching me at CVPR. βHey, I am a big fan of your β¦β.- research: 10%.- Twitter threads: 20%.- YouTube videos: 30%.- meme: 40%.
11
8
651
The privilege of having a PhD: . I can always say "I didn't get what you were saying." .and everybody will not assume that I am an idiot.
16
34
637
I will never get over how AI/ML people use the word βinferenceβ.
4
70
630
I once received advice: . "If your current project will not change the field, then it's not worthwhile doing it.". This was the time when I first started exploring research. Looking back, this is probably the MOST TERRIBLE advice for me.
9
54
604
How to create a good table?. While in grad school, I thought my job writing the paper was done after dumping all the numerical numbers from my experiments in a table. π€¦ββοΈ . Check out some tips that will help you improve the quality of your tables! π§΅.
16
120
611
Are you a student from underrepresented groups and applying for graduate school in CS this year? . Check out many wonderful (student-led) pre-application programs for supporting PhD applicants!.
16
224
601
How to share your progress with your mentors/collaborators?. Throughout your research project, 99% of the time your approach DOESN'T WORK (yet). π¬. How could we share these "failed results" and have productive conversations with your mentors/collaborators? π.
10
136
596
Excited to share that my group and I will be moving to CS dept. at University of Maryland @umdcs !. I am grateful for my mentors, students, collaborators, family, and friends for their help and support along the way. π.
55
10
584
How to drive your research forward? . βI tested the idea we discussed last time. Here are some results. It does not work. (β¦ awkward silence)β . Such conversations happen so often when meeting with students. How do we move forward? . You need β¦.
5
64
579
How to present a line plot?. Line plots are effective for describing the relationship between two variables of interests. Unfortunately, most junior students would simply copy&paste the figure from the paper in their talk and cause much confusion. π. Let's break it down . π§΅
6
111
567
*Hook*. β"Hi, my name is . I am a . from . My research . I am happy to be here. Today I am going to present this title on the slide." . β
Start your talk with a story, a picture, a surprising statistics, a quote, a question, a poll from audience, or a guessing game.
4
33
530
How to decide what to work on?. So many exciting ideas, so little time! How can one narrow down to a specific idea to work on? π€. Sharing some strategies I found useful. π§΅.
5
90
549
First Principles of Computer Vision by Shree Nayar. In the era of deep learning everything, understanding the fundamentals is more important than ever!.
3
121
537
How to do experiments?. Junior students often feel stressed before the weekly meeting with their advisors because their experiments do not go well. π©π°π±. Some tips on why, what, and how to do experiments. π§΅
10
143
537
So You Think You Want to Apply for a PhD. The application cycle is coming up. How do I maximize my chance of getting admitted to my dream program? π€. Check out the thread below for resources/advices/tips!.
17
129
528
Grad students working on their papers while the world is on fire
6
32
499
How to work with your senior advisor(s)?. Many students find it challenging to navigate grad school when working with senior professors as they are often extremely busy and hands-off in research. Check out below for some tips. π§΅.
8
97
510
I still remember I marveled at how simple and elegant the Lucas-Kanade tracker is when I first learned about it as a grad student. Here is a fun story about it. "Newness itself is not a virtue, usefulness is." - Takeo Kanade
7
95
504
How to handle questions in a presentation?. When I was a student, I always feel stressful during the Q&A session after my presentation. π΅βπ« In many cases, I didn't know how to respond well and just froze up there! π₯Ά. Sharing some tips I learned from others. π§΅.
7
95
505
How to write math in a paper?. Math allows you to convey your idea precisely and concisely. But how to write them clearly? π€. Check out some high-level tips (with examples). π§΅.
13
95
484
The IEEE PAMI editor added "China" to my co-authors from Taiwan (without consulting with us nor our consent).
22
137
470
How to improve asynchronous communication? . Two years into remote work, asynchronous communication has never been this important for work efficiency and productivity. Some tips for effective async communication π§΅.
7
84
469
How to get a tenure-track faculty job?. Interested in pursuing an academic career? Summer is the best time to kick off your search!. Here are some tips that I learned. π§΅.
4
87
475
The wait is over! . @gaochen315 released the video completion code in @GoogleColab. . No environment setup! Just run the code directly in your browser. Try it out and have fun!
The Google Colab of our video completion is up! Now you can edit the video (e.g. removing objects, FOV extrapolation) with one click!.
5
100
471
How to present a poster at a conference?. Poster presentation is a great way to disseminate your research findings, have in-depth discussions, and expand your networks! . Here are some tips for presenting like a pro! π.
4
97
470
Academics: "You should finish your PhD with three papers that you are decidedly passionate about.". Job market: "Minimum requirement: 8 top-tier conference papers for research scientist roles.".
Since we just wrapped up an AI megaconference, it felt like a good day to plead for fewer papers.
15
37
462
Me: Reviewing CS PhD/internship applications. Also me: Yep, I am absolutely sure that I will not get into any graduate programs and would get zero internship offers if I were the applicant now. Sooooo many talented candidates!.
13
13
456
Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! . How?.π‘ The good old epipolar constraints in a pose-guided diffusion model! . Paper: Project:
8
61
425
New YouTube video!. Diffusion models are awesome! But how do they actually work? π€. Check out the video and learn with me!.
4
82
432
Somehow google scholar is really concerned about a robot reading too many papers
10
10
430
Glad to learn that my silly Twitter threads are helpful! βΊοΈ
10
3
426
A bit lagging. but that OpenAI's o3 model seems incredible! . Where can I learn more about how this works, e.g., test-time compute scaling and RL CoT? Is there relevant literature?.
26
23
428
How I Understand Flow Matching. Flow matching is AMAZING! It combines the advantages of scalable training in diffusion models and flexibility in continuous normalizing flows. Check out my video and learn more about this fascinating topic!.
8
59
422
Student: I want to pursue an academic career. Do I have to do a PhD at a top university?. Me: Of course not! You will be evaluated based on the quality of your work, NOT the university prestige. Student: But all the faculty candidates we interviewed are from top places. Me:
14
27
414
For the first time in five years, I have a full-time work schedule. I thought I will trivially be 3-4 times more productive, but it turns out managing time well is so damn challenging. π¬. After watching productivity videos to procrastinate, here are what I found useful. π§΅.
2
64
397
How to organize your talk?. Qual/prelim/defense/conference season is coming up! π± How should we organize the talk so that your committee members won't fall asleep? . Some simple tips I found useful . π.
13
84
394
Writing Cold Emails. Writing emails to a stranger could be daunting, but it's a great way to build connections, explore opportunities, and even advance your career. How do we write an effective cold email? . Check out 𧡠below for some ideas.
12
86
394
Understanding ML/CV papers π°. β’ Ground truth label: .Some guy says so. β’ Learning from unlabeled data: .Learning from carefully curated ImageNet and pretend that we don't know the labels. β’ Parameter empirically determined:.Tried many paras and this has the best number.
7
40
383
CVPR: We welcome new students!. Students: Cool! I just submitted my very first paper!. CVPR: You now also need to review papers. Students: Not sure how to do that. Here is my attempt. CVPR: Your reviews are too unprofessional. You are now banned for two years. Students: . .
Voting is now closed for the motions proposed at the #CVPR2022 PAMI-TC meeting. All motions have passed:. Motion 1: Withdrawn Submissions Will No Longer Be Made Inaccessible to Reviewers.Yes: 867.No: 354. Motion 2: Making Authors Responsible for Reviewing.Yes: 677.No: 553.
14
45
386
Finished my ML course syllabus this Spring! Am I doing this right?
16
21
371