[📷] 240715 2024 IU HEREH WORLD TOUR CONCERT IN NEWARK 드디어 기다렸던 뉴애나를 만나러❤️ #아이유 #IU #HEREH #IU_WORLD_TOUR

Our new model is state of the art for voice cloning. You can try it in preview now on our playground and API under Sonic Preview*. The cloning quality is very impressive, check out this sample of my cofounder Albert Gu, cloned using 7 seconds of audio from a talk he gave recently. The voice matches perfectly, and we've now advanced our models to mimic a whole host of vocal characteristics (pitch, timbre, intonation, prosody, accent, disfluencies) and background information (soundscape, noise). We'll put out a whole host of (very) fun examples soon. The step change comes from advances we've made in helping models use new information and context at test time - in-context learning for audio models. We're now exploring ways to use the advances as part of fully multimodal models. We're also shipping some other models that use this new technique (in beta with a few customers, we'll announce soon). -- Looking ahead, you'll soon be able to put in as much audio, video and text into our models as you like. You'll be able to ask the model to mimic, change or constrain to any information present in context. "Translate clip 1 to Spanish, use clip 2's accent but slow down the speaking rate." "Dub video 1, clean up the background music in the audio, but don't touch the intro jingle. Also add in clip 2 music as outro, and clean up my background audio." "Take this script, narrate it in a light, fun voice, throw in a couple of ads from this this folder of ads in the best places." This will lead to new interfaces for creation that help users take advantage of all of the information they have in their hands. -- *You'll need a paid account to get started with cloning.

👀 @dallasmavs

who do you guys think is the best at naming their AIs? 🥰