Today, we’re excited to announce the general availability of the Contextual AI Platform. This is the first enterprise platform designed for building specialized RAG agents to support expert knowledge work. What is a specialized RAG agent? First, a general-purpose AI agent is one designed to automate simple daily tasks like scheduling a meeting or responding to an email. On the other hand, a specialized RAG agent is one designed to augment subject-matter experts performing complex domain-specific work. The Contextual AI Platform allows you to create these agents easily and achieve SOTA accuracy right out of the box. Check out what the Contextual AI Platform can do.

Super excited to share something we’ve been working on for a long time: the @ContextualAI platform is now generally available! SOTA performance across the RAG stack, for each individual component as well as end-to-end 🧵👇

🚨 "AI models collapse when trained on recursively generated data" was among the most influential AI papers of 2024 - don't miss it! Bookmark & download it below. Interesting quotes: "The development of LLMs is very involved and requires large quantities of training data. Yet, although current LLMs2,4–6, including GPT-3, were trained on predominantly human-generated text, this may change. If the training data of most future models are also scraped from the web, then they will inevitably train on data produced by their predecessors. In this paper, we investigate what happens when text produced by, for example, a version of GPT forms most of the training dataset of following models. What happens to GPT generations GPT-{n} as n increases? We discover that indiscriminately learning from data produced by other models causes ‘model collapse’—a degenerative process whereby, over time, models forget the true underlying data distribution, even in the absence of a shift in the distribution over time" - "Our evaluation suggests a ‘first mover advantage’ when it comes to training models such as LLMs. In our work, we demonstrate that training on samples from another generative model can induce a distribution shift, which—over time—causes model collapse. This in turn causes the model to misperceive the underlying learning task. To sustain learning over a long period of time, we need to make sure that access to the original data source is preserved and that further data not generated by LLMs remain available over time. The need to distinguish data generated by LLMs from other data raises questions about the provenance of content that is crawled from the Internet: it is unclear how content generated by LLMs can be tracked at scale. (...)" ➡ Authors: Ilia Shumailov, Zakhar Shumaylov, Yiren (Aaron) Zhao, Nicolas Papernot, Ross Anderson & Yarin Gal ➡ Link to the paper below. 🔥 To stay up to date with the latest developments in AI policy, compliance & regulation, including excellent research, join 44,400+ people who subscribe to my AI newsletter (link below).

One of our most exciting results: when humans evaluate LLM outputs using natural language unit tests instead of traditional preference judgments, inter-annotator agreement jumps from 71% to 86%! As LLMs continue to improve, we need more principled ways to identify flaws and measure quality. Breaking down the complex notion of "good responses" into explicit, testable criteria is crucial for this - and our results validate this across multiple studies. Throughout our paper we demonstrate this benefit of decomposition, but it's especially clear in the human annotation results. Being able to reliably measure and agree on model behavior will be key to systematic improvement in LLM capabilities 📊