Hao-Wen (Herman) Dong 董皓文
@hermanhwdong
Followers
1K
Following
839
Statuses
267
Assistant Professor at University of Michigan | PhD from UC San Diego | Human-Centered Generative AI for Content Creation
USA/Taiwan
Joined April 2020
🎉Super excited to share that our TeaserGen project led by @WeihanCHsu has been accepted to #ICLR2025! 🔍We explored a new task of generating teasers for long documentaries. 🤩We presented a new dataset, new models, and new evaluation metrics for teaser generation.
🥳 Our paper, "TeaserGen: Generating Teasers for Long Documentaries," has been accepted at #ICLR2025. In this paper, we introduce a new dataset, DocumentaryNet, propose TeaserGen systems to generate teasers, and introduce new evaluation metrics to evaluate this new task.
2
12
66
RT @BBCPolitics: "Somebody’s getting paid, so why shouldn’t it be the guy who sat down and wrote 'Yesterday'?" Sir Paul McCartney tells #B…
0
416
0
Very excited about the many potential applications and extensions of this project! How about...🤔 🎶adding music & sound effects? 🎬making movie trailers? 🛒making commercials for products? 🧑🏫creating an AI learning agent that summarizes lecture recordings & educational videos?
🎵 Want to make our teasers even more captivating? How about layering in some music tracks? 🎬 Can we leverage our (publicly available soon) dataset as another lens for AI Filmmaking? 📚 Educators, what if we made short, engaging intro videos to enhance the learning experience?
0
3
18
0
0
2
RT @deeplearnmusic: 🧑🎓Our @ISMIRConf Tutorial "Deep Learning 101 for Audio-based MIR" provides a broad introduction to music audio proces…
0
13
0
Incredible audiovisual alignment!!
🎥 Introducing MultiFoley, a video-aware audio generation method with multimodal controls! 🔊 We can ⌨️Make a typewriter sound like a piano 🎹 🐱Make a cat meow like a lion roars! 🦁 ⏱️Perfectly time existing SFX 💥 to a video
0
0
4
Pretty cool!
🚨Thrilled to share my first PhD project: DreamRunner✨: Fine-Grained Storytelling Video Generation with Retrieval-Augmented Motion Adaptation ➡️ A novel storytelling video generation framework based on retrieval-augmented motion prior learning and spatial-temporal region-based 3D attention and prior injection module. ➡️ Capable of producing consistent, multi-motion, multi-character storytelling videos across multiple scenes. ➡️ Achieves strong fine-grained condition-following for compositional text-to-video generation, not only improving the base model in all dimensions on T2V-CompBench, but also pushing the performance boundary of open-sourced models to match or outperform commercial models in 3 out of 6 dimensions on T2V-CompBench. Thread 🧵👇
1
4
9
The next three grand challenges of @ISMIRConf proposed by @MasatakaGoto! Let's work on it! 🤘 - Ultimate Music Retrieval: Query-by-Brain 🧠 - Ultimate Music Creation: Music Drug 💊 - Ultimate Music Listening: Direct Digital Music 🎵
0
3
48
Come chat with us at @ISMIRConf! Jiwoo Ryu, Hao-Wen Dong, Jongmin Jung, and Dasaem Jeong, "Nested Music Transformer: Sequentially Decoding Compound Tokens in Symbolic Music and Audio Generation," ISMIR, 2024. 📜Paper: 🎵Demo:
Our paper, Nested Music Transformer, got accepted for @ISMIRConf!!🎉 We propose a new architecture for LM on compound tokens (note-level symbolic or RVQ audio tokens). Work by master's student Jiwoo Ryu and @sake_min, with @hermanhwdong! #ismir2024 Demo:
1
5
36
Proud to be an alumnus of the amazing AI x Music group at @UCSanDiego 🤘 Go Tritons 🔱
We’ve just launched our official Twitter (X) account! We’ll be sharing exciting AI x Music projects from UCSD (PIs: @McAuleyLabUCSD @BergKirkpatrick and Shlomo Dubnov) Stay tuned for updates on our ISMIR participation!
0
1
24
RT @elthateng: Happy to share that I am organizing an ALMA data reduction workshop on Oct 29 at @UMDAstronomy, supported by @TheNRAO! Anyon…
0
12
0