🤖 🗞️ Free AI Generated Digital Humans Newsletter -> Like everyone else, I've been struggling to keep up with the sheer volume of papers and news in the Digital Human space. Over the past few weeks, I've developed an agentic (ish) AI pipeline to find and

🏎️ We're starting to see some real-time diffusion models for Avatar generation!. RAP: Real-time Audio-driven Portrait Animation with Video Diffusion Transformer. TLDR: This DiT model is able to perform audio-to-talking head synthesis in real-time. The key is the use of a very

🎶 👗 ByteDance are making amazing progress for Virtual Try-On!. DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Framework. TLDR: This work overcomes the reliance on hard-to-find paired (video, garment) data. By using a

📣 Accepted Paper 📣. I'm happy to share that our work DEAD: Data Effecient Audiovisual Dubbing using Neural Rendering Priors (previously Dubbing for Everyone) has been accepted to BMVC25!. TLDR: We achive high-quality dubbing with just 4 seconds of personalised data. We seperate

🛫 A double whammy from Microsoft this week, with another synthetics-based paper. This time, you can lift a video stream to 3D with free-viewpoint rendering that tracks the viewer!. VoluMe – Authentic 3D Video Calls from Live Gaussian Splat Prediction. TLDR: A UNET-based model

🧑‍🦳 👩‍🦳 Meta's real-time Codec Avatars now have hair control. Imagine being able to customise your hairstyle in VR!. HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars. TLDR: Trains two separate hyper-networks, one for the face and one for the hair. The data to do

⏱️ Turn yourself into a 3D Avatar in real-time with StreamME from Adobe and the University of Rochester (code coming soon). StreamME: Simplify 3D Gaussian Avatar within Live Stream. TLDR: This work speeds up Gaussian reconstruction using motion-aware anchor points to prevent the

Want to get up to three papers like this directly to your inbox and summarised by AI each day? You can sign up for free here:

🔥 Microsoft once again showing off the power of synthetic data for human-centric computer vision! Showing how you can use it for:. - 🧐 Depth Prediction.- ↗️ Normal Estimation.- 👤 Background Segmentation. Synthetic data has pixel-perfect label accuracy and is nearly

SoulDance: Music-Aligned Holistic 3D Dance Generation via Hierarchical Motion Modelling. TLDR: A large dataset and evaluation framework (with new metrics) for 4D dance generation from music audio. 📽️ Project Page: 📜 Paper:

SnapMoGen: Human Motion Generation from Expressive Texts. TLDR: This is a method for animating digital characters from text. The model uses multi-scale tokenisation with a masked generative transformer. It is trained on a novel dataset (which should be released) consisting of 44

Want to get up to three papers like this directly to your inbox and summarised by AI each day? You can sign up for free here:

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions. TLDR: A method for Gaussian Avatars which improves the ability to capture fine details by using a number of patches and a small MLP per patch to do independent expression-conditioning, rather than a single

Want to get up to three papers like this directly to your inbox and summarised by AI each day? You can sign up for free here:

Digital Salon: An AI and Physics-Driven Tool for 3D Hair Grooming and Simulation. TLDR: A tool for modelling hair, with a text interface, physics simulation and high-quality rendering system. With code released!. 📽️ Project Page: 📜 Paper:

Want to get up to three papers like this directly to your inbox and summarised by AI each day? You can sign up for free here:

VisualSpeaker: Visually-Guided 3D Avatar Lip Synthesis. TLDR: Improves speech-driven facial mesh animation by rendering with 3DGS Avatars and running the results through a lip-reading network to get lip sync as a loss. 📜 Paper:

Want to get up to three papers like this directly to your inbox and summarised by AI each day? You can sign up for free here:

HyperGaussians: High-Dimensional Gaussian Splatting for High-Fidelity Animatable Face Avatars. TLDR: Lifts Gaussians to higher dimensions, which can then be conditioned for the problem of Gaussian Avatars. This enables much better reconstruction of high-frequency details and

🤖📰Want to get up to three papers like this directly to your inbox and summarised by AI each day? You can sign up for free here:

ARIG: Autoregressive Interactive Head Generation for Real-time Conversations. TLDR: A method for dyadic conversations (speaking and listening) in real-time. This work uses an autoregressive diffusion MLP with added signals from context learning (by fusing LivePortrait parameters