Humanloop is now generally available! After 2 years of working closely with early customers, we're opening access to our full evals platform. 🧵 Here's what we've learned and how we can help you build great AI products:

When do you know it's time to try fine-tuning instead of prompt engineering? Our CEO @RazRazcle is on Data Radicals with @satyx this week to discuss: 🔹 How fine-tuning tends to be an optimization step, which comes once you've pushed the limits of prompt engineering 🔹 Why collaboration with domain experts in the AI product development cycle is key to driving successful outcomes 🔹 How software engineering is changing in the age of AI And lots more! Watch the full episode here:

🇬🇧 London - come to our first AI Product Management Meetup on Tuesday Feb 18th! Meet with fellow AI product leaders and enjoy food and drinks on us in Bloomsbury. We’re hosting a panel featuring guests who are building world-class AI products, sharing their learnings, followed by a social event for all attending. RSVP (space limited):

Read more:

o3-mini now available in the prompt editor — with streaming!

@garrytan and its going to be driven by product people (agency) and subject matter experts (taste)

@RuiCarrilho5 May your timeline be blessed with fewer regrettable minutes.

@KyeGomezB @swarms_corp only if you want to add a lot of noise to your timeline!

@0PointAlchemy patience, sir.

RT @garrytan: Prompt (agency) and evals (taste) will rule everything around us

Register here:

Read more on updates:

“We strike a balance between automation and human expertise, ensuring people stay at the heart of creative decisions” - Nikesh Hotchandani, AI Product Owner at Tag Nikesh joined us in London to talk about how he’s deploying AI in a responsible and scalable manner for one of the world’s largest marketing production agencies. He spoke about how LLMs are transforming marketing but in order for his team to scale AI production, they must first ensure model output is aligned with Tag’s global standards. To solve this, Nikesh and his team made evaluations a core part of their workflow - and now they leverage Humanloop to ensure all of their prompts align with company guidelines and perform responsibly We’re proud to be supporting Nikesh’s team at Tag! Thanks for coming by.

RT @Tom_MkV: @humanloop @AppearAPI I'm wiping Hoosh's db in Appear, then firing all the endpoints by clicking through the UI, which then au…

“You wouldn’t build a $100m software product without unit tests. Then how can you think of building a $100m AI product without evals?” - Noam Rubin, AI Platform team at @TrustVanta Noam joined us in SF to speak about how his team have used Humanloop to build some of the most compelling AI products on the market. Noam spoke about the differences between traditional software development and building with AI. "Most engineers haven't built with stochastic software before, and so teaching them about how to use evals and datasets in iterative deployment has been key" Noam's team use Humanloop to run evaluations, which is now part of their CI/CD workflow. "We don't ship a prompt change now unless it has an eval report from Humanloop. Its literally in the PR" Thanks for coming by Noam! We're stoked to be supporting you.

Today, our CTO Peter Hayes reflects on progress made in 2024, and how this process has enabled us to set a new standard for enterprise-grade AI engineering. Read here: