🥳 Introducing SpeechGPT 2.0-preview: A GPT-4o-level, real-time spoken dialogue system! (Only Chinese for now) 🎆 Highlights: ～⚡️ Real-time speech-to-speech dialogue with latency under 200ms ～😊 Rich in emotion and diverse in style, with strong speech style generalization ～🦁 Strong role-playing capabilities 🤖️ Try it out: Online system: Github: More demos:

AnyGPT Unified Multimodal LLM with Discrete Sequence Modeling introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music. AnyGPT can be trained stably without any alterations to the current large language model (LLM) architecture or training paradigms. Instead, it relies exclusively on data-level preprocessing, facilitating the seamless integration of new modalities into LLMs, akin to the incorporation of new languages. We build a multimodal text-centric dataset for multimodal alignment pre-training. Utilizing generative models, we synthesize the first large-scale any-to-any multimodal instruction dataset. It consists of 108k samples of multi-turn conversations that intricately interweave various modalities, thus equipping the model to handle arbitrary combinations of multimodal inputs and outputs. Experimental results demonstrate that AnyGPT is capable of facilitating any-to-any multimodal conversation while achieving performance comparable to specialized models across all modalities, proving that discrete representations can effectively and conveniently unify multiple modalities within a language model.

Google Research: Introducing Pathways, a next-generation AI architecture

We're excited to announce that Yikang Shen, Shawn Tan, Alessandro Sordoni @murefil and Aaron Courville received the Best Paper Award at @ICLR2019. Discover their work on Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks: #ICLR2019