super excited to share what we've been cooking up at
@haizelabs
🕊️🕊️
we are now in the era of grossly excessive AI hype and demoware.
but it is high time to recalibrate and revisit the difficult, unsexy, underlying problem that everybody is avoiding -- the AI reliability and
Today is a bad, bad day to be a language model.
Today, we announce the Haize Labs manifesto.
@haizelabs
haizes (automatically red-teams) AI systems to preemptively discover and eliminate any failure mode
We showcase below one particular application of haizing: jailbreaking the
commoditized LLMs as a judge are not a panacea for eval
automated evals will only ever be as good as the data you configure them on
for that reason, we are stoked to introduce Sphynx, a fuzz-tester for surfacing challenging, hallucination-inducing questions. these questions
1/ introducing Sphynx - the leading hallucination haizing algorithm🕊️😼
- breaks SOTA hallucination detection models (HDM)
- open source, open data
- surfaces critical hallucinations in high-stakes domains
- enables adversarial training for more robust hallucination detection
playing around with Command-R today -- actually insanely impressive. good tool use, low hallucination, great retrieval
*this* is the language model enterprise customers actually need
well done
@cohere
🥳
🚀 excited to share our latest
@NeurIPSConf
work highlighting the fundamental gap between human and machine
#vision
🔍 we revive decades-old cognitive science concepts and apply it to neural networks, studying their ability to classify degraded polygons: a basic yet telling task
cooked up a lil something something with
@steveshenli
,
@clefourrier
, and the amazing
@huggingface
team
in short: we carry out a comprehensive + unified study of language models' red-teaming resistance
blog:
code:
Models are being deployed in real life situations... but are they safe?
The Red-Teaming Resistance leaderboard tests if models resist harmful instructions (fake news creation, malware diffusion, harassment, etc) 🔥
Super useful, congrats to
@haizelabs
!
if DSPy has a million fans, then i am one of them.
if DSPy has ten fans, then i am one of them.
if DSPy has only one fan then that is me.
if DSPy has no fans, then that means i am no longer on earth.
if the world is against DSPy, then i am against the world.
🕊️red-teaming LLMs with DSPy🕊️
tldr; we use DSPy, a framework for structuring & optimizing language programs, to red-team LLMs
🥳this is the first attempt to use an auto-prompting framework for red-teaming, and one of the *deepest* language programs to date
so by this point it is pretty clear who got into EE/CS PhDs and did not
it was an ridiculously competitive year
many of my friends with *truly* insane accomplishments and talent still didn't get in...
if you are in this position, please DM me -- i have something of interest
shoutout wolfe tone’s pub and bartender for facilitating the biggest research breakthrough of my life
was stuck on this problem with ZERO progress for months, but now everything is falling into place
we’re so back.
🕊️hello nyc ai enthusiasts :^)
🤫
@haizelabs
and i are hosting a super super sneak peek of our ai haizing (i.e. red-teaming x fuzzing) platform tomorrow night as part of
@TechweekNYC
#NYTechWeek
👀people will get private access to our product to haize whatever model they like!
update -- the sales agent i built to do outbound emails went too ham:
dm me if you want access to the agent, or i might just open source it cuz why not
also soliciting names for this agent...maybe Salena? open to suggestions
for whoever is sending me leads, please send me something slightly more reasonable than the following (the bed...under the bed):
nyc housing is tough but not that tough
Meet Sohu, the fastest AI chip of all time.
With over 500,000 tokens per second running Llama 70B, Sohu lets you build products that are impossible on GPUs. One 8xSohu server replaces 160 H100s.
Sohu is the first specialized chip (ASIC) for transformer models. By specializing,
Super cool application of
#DSPy
. This is the kind of stuff our team expects to happen in LM-based development -- more automated search over everything from prompts to pipeline designs.
🚨Announcing the largest study focused on *how* to optimize the prompts within LM programs, a key DSPy challenge.
Should we use LMs to… Craft instructions? Self-generate examples? Handle credit assignment? Specify a Bayesian model?
By
@kristahopsalong
*
@michaelryan207
* &team🧵
if you didn't already know,
@GroqInc
is killing the inference game, thanks to its dedicated LPU hardware.
185 tokens/sec vs.
@anyscalecompute
at 66 tokens/sec for 70B Llama-2.
it's actually insane how fast this is, try it out:
Haize Labs Introduced Sphynx: A Cutting-Edge Solution for AI Hallucination Detection with Dynamic Testing and Fuzzing Techniques
Haize Labs has recently introduced Sphynx, an innovative tool designed to address the persistent challenge of hallucination in AI models. In this
a quick sunday afternoon read from the blog :^)
everybody is interested in red-teaming, but what should we do to actually *defend* against vulnerabilities that are discovered?
we lay out practical and promising defense methods here:
there's been a lot of great work about benchmarks for agents (e.g. AgentBench, WebArena, etc.)
but who's thinking about agents *AS* benchmarks? in this new era of hyper-dynamic systems, static datasets probably aren't going to cut it
this seems criminally under explored...
I 💙 my CS HCI grad-school course last semester where we read & discussed research papers every week
SO my friends
@haizelabs
& I are hosting a NYC AI Research Paper Reading Group on 6/11 from 5:30-7:30 for 20 academics and industry folks working in and/or interested in AI 🪄📖
this is a cool jailbreaking method
but the fact that the microsoft azure cto sat around playing with chat interfaces and wrote a prompting paper is amusing af to me
bootstrap means something totally different if you talk to compiler vs. business vs. statistics people
still not as overloaded as kernel lol (algebra vs. ML vs. OS vs. signal processing, etc.)
Automated redteaming evals can be very fallible !!!
Just got 26% ASR on the HarmBench val set, using their provided Mistral-7B val classifier, simply by regurgitating + noising the prompts as outputs.
This is the entire setup, with some example outputs:
💉you've heard of Needle in a Haystack, now get ready for Thorn in a HaizeStack
👀tldr => a jailbreak text ("Thorn") embedded in a wall of distractor text ("HaizeStack") easily circumvents GPT-4's (and other) safeguards.
🤧try harder guys!
🧑💻code here:
just met one of the most passionate -- but straight up degenerate --- guys building in consumer social. he's doing a random dare rn b/c somebody on his app paid him $5 to do so.
in a former life he was on child genius (the show) lmao
Yesterday was the 100th anniversary of the first human EEG recording by Hans Berger.
Today we’re excited to introduce Hans🧠, the world’s first affordable, research-grade, non-invasive mind reading EEG headset.
Hans can be used for a variety of EEG tasks, including
AI/ML phd admissions are ridiculous
In undergrad I had 1 first author, 2 other accepted papers, 1 workshop co first author papers (albeit not my best work tbh) at ICLR/ICML/NeurIPS, ran one of the biggest RL competitions and still felt lucky to get offers
New paper! 🫡
We introduce Representation Finetuning (ReFT), a framework for powerful, efficient, and interpretable finetuning of LMs by learning interventions on representations. We match/surpass PEFTs on commonsense, math, instruct-tuning, and NLU with 10–50× fewer parameters.
Introducing "Red Teaming in Public" 🚨
@redteaminpublic
With great power comes great responsibility, but today, many AI startup apps lack even basic safeguards against criminal abuse
Here's how some friends including
@pablothee
& I aim to fix it, and how you can help
🧵👇
went to my first web3 conference this weekend.
somehow almost nobody is seriously thinking about the fundamental and unsolved issue of transaction security
instead, people get rewarded for making…NFTs that interact with each other?
make it make sense
🚨 Are leading safety-aligned LLMs adversarially robust? 🚨
❗In our new work, we jailbreak basically all of them with ≈100% success rate (according to GPT-4 as a semantic judge):
- Claude 1.2 / 2.0 / 2.1 / 3 Haiku / 3 Sonnet / 3 Opus,
- GPT-3.5 / GPT-4,
- R2D2-7B from
⁉️...but on a dataset of 18,250 harmful jokes from Reddit -- that directly violate OpenAI's, Google's, and Meta's usage policies -- these APIs perform abysmally.
No API is able to flag even more than 50% of these harmful texts!
(3/4)
last thursday, Meta dropped Llama 3, the OpenAI killer. no doubt a very impressive model!
but over the weekend, we discovered an extremely trivial programmatic jailbreak against llama 3...sorry zuck!😘
so much for all that safety-tuning☹️
code:
@SanderSchulhoff
thanks a ton for your support Sander!!
still remember when we first met and you were incredulous that we got jailbreak GPT😆
collab soon!!