Very excited to release our Model Hub 😍 - a collection of LLM endpoints with live runtime benchmarks all plotted across time 📈 We currently have 21 models provided by: @anyscalecompute, @perplexity_ai, @replicate, @togethercompute, @octoml, @MistralAI and @OpenAI, with many more on the roadmap. We test across different regions (Asia, US, Europe), with varied concurrency and sequence length. By plotting across time, our dashboard highlights the stability and variability of the different endpoints, and their ongoing evolution across API updates and system changes. Our benchmarking code is open source: Following the great work from @withmartian last week, we mention several new findings in the thread below, focusing on llama-2-70b-chat and mixtral-8x7b-instruct-v0.1 ⬇️ Our unified API also makes it very easy to test and deploy these different endpoints in production, without needing to create several accounts 🔑 Our Hub is a work in progress, and we will be releasing new features every week 🚀 We are granting everyone $5 starting credits with a free top up of $2.50 every week, compatible with all major LLM providers (more coming soon!). You can sign up here Please tell us what you think, and we’ll quickly incorporate feedback into the next weekly release! 😊 As always - let’s unify AI! 🟢💪

RT @ivy_llc: We’re excited to announce Ivy is partnering with Kornia, allowing Kornia to be used with TensorFlow, JAX, and NumPy for the fi…

RT @DanielLenton1: Incredibly flattered that @amazon have invited me to be their keynote speaker for this year's AWS Gen AI Loft event. Can…

RT @ParasMadan9: Open AI released the new model O1 and I tested and compared it's logical thinking capabilities with Claude 3.5 Sonnet usin…

We’re excited to have @shirleyxiaoyic from @IndianaUniv, co-author of the paper "The Janus Interface: How Fine-Tuning in Large Language Models Amplifies Privacy Risks," joining us this Friday for our Paper Reading Session! 🤩 RSVP 👉 The research introduces a novel attack, Janus, which exploits the fine-tuning interface to recover forgotten PIIs from the pre-training data in LLMs also formalizing the privacy leakage problem in LLMs, explaining why forgotten PIIs can be recovered through empirical analysis on open-source language models.🧠 Check out the Paper: See you there!

We're thrilled to announce that @Vapi_AI will be joining us for our weekly Webinar Series tomorrow! (Wednesday) 🤩 RSVP here: Join us as we welcome Sahil Suman, Solution Engineer at Vapi AI, to the session. Discover how VAPI enables the quick setup of high-quality voice agents and see the integration of @letsunifyai with VAPI for seamless access to various models and providers. See you there! 🧑‍💻 Explore Vapi: ⚡️ ⚡️

We are really excited to welcome @zlwang_cs from @ucsd_cse, who co-authored the paper "Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting". Happening Tomorrow!🤩 RSVP 👉 The research introduces SPECULATIVE RAG, a framework that leverages a larger generalist LM to efficiently verify multiple RAG drafts produced in parallel by a smaller, distilled specialist LM 🧠🤖 Check out the Paper👉 See you there!

We are really excited to announce that we will be joined by @llmware for our Webinar Series today! (Tuesday)🤩 RSVP👉 In this session, we're excited to welcome Darren Oberst and Namee Oberst from LLMware. We will explore how small specialized LLMs can compete with the larger models for specific use cases, especially for Financial, Legal, Compliance, and Regulatory-Intensive Industries. See you there! 🧑‍💻 Checkout LLMWare: ⚡️ ⚡️

We are really excited to welcome Devichand Budagam from @IITKgp, who co-authored the paper "Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models". Happening Friday! 🤩 RSVP👉 ​The research introduces Hierarchical Prompting Taxonomy (HPT), a universal evaluation metric that can be used to evaluate both the datasets' complexity and LLMs' capabilities🧠 Check out the Paper👉 See you there!

We are really excited to announce that we will be joined by @tavilyai for our Webinar Series this Tuesday!🤩 RSVP👉 In this session, we'll explore how Tavily API makes search engine optimised for LLMs and RAG, to provide efficient, quick, and persistent search results. We'll also showcase Unify's SSO integration with Tavily 🧠🧑‍💻 Checkout Tavily: ⚡️ ⚡️

We are really excited to welcome @sh_reya from @Berkeley_EECS, who co-authored the paper "Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences". Happening Tomorrow! 🤩 RSVP 👉 The research introduces EvalGen, an interface that provides automated assistance in generating evaluation criteria and implementing assertions🧠👩‍💻 Check out the Paper👉 See you there!

We are really excited to announce that we will be joined by @CopilotKit for our Webinar Series this Thursday!🤩 RSVP 👉 In this session, we'll explore how CopilotKit makes integrating AI-powered Copilots into any product easy. We'll also showcase how the Unify Router can help you create Copilots that automatically select the best Large Language Model (LLM) for the task 🤖✨ Checkout CopilotKit: ⚡ ⚡ See you there!

We are really excited to welcome @yyzTodd from @ZJU_China, who co-authored the paper "Knowledge Circuits in Pretrained Transformers". This Friday! 🤩 RSVP 👉 The research dives into the computation graph of language models to uncover the knowledge circuits that are instrumental in articulating specific knowledge 🤖🧠 Check out the Paper 👉 See you there!

LLM Contributor Projects Week - LLM Resume Analyser 🔍 ⚡️Try it out! - Oscar Arroyo Vega (@OscarArroyoVega), Sanjay Suthar (@Sanjay__Suthar), Jeyabalan Ganesh Nadar, and Maissa Rekik have been working on an LLM Resume Analyser. The app extracts and analyzes key skills from resumes, provides matching scores for job descriptions, and suggests resume improvements! 📚✨ Check out their presentation - Check out the repo here -

We are super excited to welcome @felipemaiapolo from @UMich this Friday to our paper reading session. Felipe co-authored the paper "Efficient multi-prompt evaluation of LLMs" 🤩 ⚡️RSVP - His research introduces PromptEval, a new method for estimating performance across a large set of prompts, borrowing strength across prompts and examples to produce accurate estimates under practical evaluation budgets📈 📚Check out the paper - See you there!

LLM Contributor Projects Week - LlamaIndex RAG Playground🛝 Abhijeet Chintakunta has developed the LLamaIndex RAG Playground, an application that allows users to upload PDF documents and ask questions about them!⚡️ Try it out - Abhijeet used LLamaIndex to create a RAG application for this project. The application's interface is built with Streamlit, making it user-friendly and easy to interact with 🤝🤖 Check out their presentation - Check out the repo here -