CEO and founder of XBOW. Previously: Founder of GitHub Next, founder of GitHub Copilot, CEO and founder of Semmle (GitHub Advanced Security), prof at Oxford.
At the RSA conference in May, I asked every CISO whether they’d use an AI web pentester if it matched a skilled human. They said: “Ha, ha! That’d be amazing! See you in five years!”
It’s here now.
XBOW is the world’s first fully automated web pentester.
It previously scored an unprecedented 75% on renowned web pentesting benchmarks from
@PentesterLab
and
@PortSwigger
.
So we decided to give it a harder challenge: competing against humans.
I quit GitHub. I’m proud of GitHub Copilot and GitHub Advanced Security (previously Semmle). Leading the creation of these products was exhilarating. 1 / 3
In my team at GitHub, we'd like to study examples of "nefarious commits" in open source, which introduce a bug on purpose. Can you point me at such commits? Could it have been detected by analysing the committer's behaviour as well as the code change itself?
Also thanks to our collaborators at OpenAI and Microsoft, with whom we built GitHub Copilot - really the first large scale example of the usefulness of LLMs, which opened the world's eyes to the value of AI. 3 / 3
Proud and humbled to lead the incredible team that created this: . We worked closely with the brilliant folks at OpenAI, and the VS Code team moved mountains to enable inline suggestions. Thanks all!
Nothing really prepared me for being a founder. Not even having done it before! But this spring Sequoia Arc helped me articulate the story of
@xbow
, and work through the key questions with like-minded experts and founders. It’s a game-changer! Join now:
Thank you, from the bottom of my heart, to the teams at
@Semmle
and
@GitHubNext
! It’s been amazing working with you, who are so brilliant and kind. I’ll eagerly follow what you build over the coming months and years. Hopefully we’ll work together again in the future. 2 / 3
My dad gave me a Pascal compiler for my birthday in 1981. It opened a new world. A few decades later, I started building stuff on open source - another watershed moment. Today, we’re launching GitHub Copilot, an AI pair programmer. This is the big one: .
I founded a new company:
@xbow
. XBOW brings AI to offensive security, augmenting the productivity of pentesters, bug hunters and security researchers.
945 days ago, on April 15, 2021, we first demoed GitHub Copilot to
@satyanadella
. It was the dawn of the age of Copilots. Where shall we be in 945 days from now?
Copilot will be the new UI for both the world's knowledge and your organization's knowledge, but most importantly, it will be your agent that helps you act on that knowledge. Here are highlights from my keynote today at
#MSIgnite
.
University professors make awesome startup employees because they brim with crazy ideas and are used to being scrappy due to tiny research grants. At
@xbow
, we have four: Andy Rice, Johan Rosenkilde,
@moyix
, and me. It works!
The GitHub Copilot team is on fire. When we imagined the Coding Oracle back in 2020, we could only dream about "Copilot Workspace"... and now it's here!
Everything I’ve done before has led up to
@xbow
- my work as a prof at Oxford, founder of Semmle / GitHub Advanced Security, and founder of GitHub Copilot:
@GitHubNext
is a unique team. They’re researchers, but they build. They’re specialist experts, but they collaborate. They are totally open-minded, and focussed on delighting users. You might find such a team in a nimble startup, but they’re at GitHub, deftly using its resources.
I didn’t expect to take external seed investment for
@xbow
. However, when
@kostabuhler
and
@laurenmhreeder
reached out on Xmas eve, we quickly discovered a common purpose, mapping out a path to change offensive security for good. Sequoia Capital is leading XBOW’s $20M seed
How did XBOW create an offensive security agent that solves 75% of web security benchmarks? With the best team and investors! Delighted that
@sequoia
is leading our $20M seed round, with participation by
@oegerikus
@amasad
@pirroh
@oliveur
& others.
Earlier this year, we partnered with
@oegerikus
as he and his team builds
@Xbow
. The company brings start-of-the-art software and AI technology to the traditionally services heavy penetration testing market. This week,
@Xbow
unveiled that they matched the performance of a human
On my way to GitHub Universe - our team (which also brought you GitHub Copilot) will launch mind-blowing fireworks. I’m most excited, however, about seeing many of my teammates in person.
@GitHubNext
#GitHubNext
has openings for machine learning researchers, ideally with experience in LLMs and/or code generation. Products not papers, over 94M users for your work, awesome team . Sounds good? Send me a DM!
Yesterday, we (
@GitHubNext
) released a new update to
#HeyGitHub
. It includes two new features that I've worked on:
* Hey, GitHub! can now generate and execute commands in the VSCode terminal.
GitHub moves as nimbly as any startup. New ideas are hatched in
#GitHubNext
, and brought to preview. When they find product-market fit, the whole org goes all-in to bring them to GA.
If there was ever a perfect application of AI to
#cyerbsecurity
, it is pen testing -- something each software organization has been throwing human time and effort into. The software of
@Xbow
, a
@sequoia
company, now does it better and faster than humans.
So proud to see CodeQL continue to grow and flourish! So grateful it is in good hands! This is what we dreamt about when we started it in December 2006.
🚀 CodeQL zero to hero part 3: Security research with CodeQL! Learn how to audit applications for vulnerabilities with CodeQL, tricks we can use for security research workflow, and how to find bugs in thousands of GitHub repos at once using MRVA.
At XBOW, we believe all claims must be presented with objective proof. So here are the web security benchmarks we used to evaluate our own system. Let us know what you think!
We are now making our validation benchmarks public! We invite you to test your skills or systems against them and share your results with us. Read more in our blog post:
What an incredibly energising GitHub Universe it has been. I loved meeting friends old and new, and learning so much from you all. But also, now happy to be back home in Malta!
Absolutely thrilled to have Brendan Dolan-Gavitt (
@moyix
) of
@NYUniversity
and
@XBOW
as our second keynote for
#FUZZING
'24 in Vienna!
Brendan's keynote will be followed by a 45min discussion on challenges and opportunities of LLMs and fuzzing for bug finding.
Peer into the future of developer tools with this magic show by
@kdaigle
. But don't be fooled - the magic is all real, here and now, made by
@GitHubNext
.
Tomorrow at 3:30 PM PT, it's time to show you the experiments and ideas from the workbench of
@GitHubNext
! Watch as I demo the future of software development TODAY via a bunch of live demos.
#GitHubUniverse
@ericabrescia
Thanks
@ericabrescia
! I so much enjoyed working with you, and I learnt a lot. Those learnings will help do it all over again, but bigger and better!
I am seeking an apprentice.
Requirements are:
•High IQ, high EQ
•Love for tech and capitalism
•High agency and low ego
•Taste
•Discretion
•A working cringe detector
•Instinct for narrative
•Strong writing ability
•Risk tolerance
•Immaculate vibes
•High-functioning
So glad we're finally working together! For one thing, now you can tell us how to do it better before it goes out :D And yes, it was hilarious you used CodeQL to assess the security of Copilot outputs!
This has been one of the most fun things about working at
@Xbow
! Remember that time I spent an absurd amount reverse engineering Copilot? Now I can simply talk to the people that wrote it :D
1/3 Following my Google departure news from last week, it is with great pleasure to announce I joined
@Semmle
as their Chief Security Officer. My duties not only cover protecting corporate assets but also building a world class open source security research team.
Semmle QL in action $95K in bounties for bugs found by
@mmolgtm
: Google Patches More High-Value Chrome Sandbox Escape Vulnerabilities | via
@SecurityWeek
Awesome results by
@helie_jean
on objectively measuring code quality with : . Here's one nugget I love: GitHub stars and quality score are highly correlated, for all languages. Try it now on your own repo!
@lgtmhq
@SemmleInc
@github
Have you heard Paganini play the violin? No? Here's your chance to see Pavel Avgustinov play QL! See the virtuoso write beautiful and useful queries, and learn how to do variant analysis of your own.
Want a technical deep dive into recent Ghostscript
#exploits
? Join our webinar "How to find type confusion vulnerabilities in Ghostscript" with
@pavgustinov
on July 24.
Microsoft's code analysis expert Michael Fanning explains how they're using
@SemmleInc
QL for DevSecOps: . Brilliant stats there: in one assessment, 3X more vulnerabilities found with
@SemmleInc
QL!
#SemmleQL
We have so much gratitude & excitement to share our
@Work_Bench
$47M Fund II with the world.
Read more from
@jerseejess
and me about how we're rethinking enterprise VC:
💯 IT to VC team
🏢 Deep F500 network
💰 Help enterprise startups close customers!
Could not have had better partners on this journey! Indeed, many of the *massive* starter deals came through
@Work_Bench
. And so much sage advice, thanks
@fendien
to you and the entire team!
What a journey w/
@oegerikus
+
@Semmle
!
-We first met in 2012 during my Morgan Stanley IT days
-Reconnected in 2014 when they expanded to NYC
-
@Work_Bench
invested + proudly witnessed Semmle close *massive* deals, many through our F500 intros
-Next up:
@natfriedman
&
@github
!
I genuinely would not have believed you if you'd told me just a few months ago that an agent could do a CBC padding oracle attack – that's probably the hardest crypto attack I teach in offsec. And yet, it hacks
Docs are for answering your questions! So GitHub Copilot reads the docs, and you can have an intelligent conversation in plain English to find what you need. An experiment by
@GitHubNext
, we’d love to hear what you think!
@idangazit
Take a break
@idangazit
! You moved mountains for the Copilot X launch, it's time to relax. I'm so grateful for all the things you did for
@GitHubNext
, and for the companionship as we grew it. Hopefully we can work together again in the future.
@brian_lovin
Thanks, Brian! I loved working with you. Remember that video with a mock demo, of an AI with a chat UX for coding, that you and I made in August 2020? It just took two years for the technology to catch up with the vision...
1/4 - How accurate are long-term projections for Covid-19? Here is a small experiment with three models: CovidSim from Imperial, CovaSim from IDM, and ModelingCovid from Stripe/Stanford:
@IDMOD_ORG
@MRC_Outbreak
Bug hunters don’t always need to reinvent the wheel. They research vulnerabilities online, and adapt known exploits to the situation in hand. I find it thrilling to see
@xbow
do that too - check the detailed workings for yourself!
Can’t find a working PoC? No problem, says XBOW—and it proceeds to read 22,000 words’ worth of GitHub issues to understand the vulnerability before writing its own exploit. Check out the trace:
7/7 Thanks to everyone on the Semmle team. What a magnificent recognition, and what a wonderful opportunity for us all! Looking forward to many more years of securing software together.
@HamelHusain
Thanks
@HamelHusain
, for the kind words, and for the inspiring discussions about the potential of machine learning, everywhere in GitHub. Was that really only two years ago?
Thanks Kelley! You and everyone else at the
@Work_Bench
have been a tremendous help to Semmle. Fellow founders, if your company is in enterprise software, you *must* work with
@Work_Bench
: their community and network is beyond incredible.
a lot of big security news coming from
@github
today
but most importantly,
@Semmle
is joining them!
it's been a fun ride with
@oegerikus
and the team. they've hired amazing talent like
@fjserna
.
excited to see what's next!
Step into the jungle by visiting our interactive hangout at
#GitHubUniverse
this week. We have TShirts and Stickers to give away, and are giving two presentations at the demo booth. Plus our competition is still going until the end of the month!
Thanks
@fendien
! At
@semmle
, we have the magic combination:
Great investors (
@Work_Bench
,
@Accel
), great team (
@fjserna
,
@kolofsen
), awesome technology, and great customers (Google, Microsoft, Uber, ...). It's a delight and pleasure to work with them all.
With
@djurado9
, we'll be discussing our validation benchmarks and results at
@BugBountyDEFCON
in
@defcon
!
Join us to learn more about it! 😊
"Leveraging AI for Smarter Bug Bounties"
📅 Saturday, Aug 10
🕝 2:30pm
📍 Creator Stage 4
We can’t wait to see you there!
Hopefully we'll see many more of these benchmark sets! At
@xbow
, we're already hard at work on our next set of benchmarks, which will be yet more realistic and challenging.
These validation benchmark have been critical for us to verify our assumptions and continuously increment our results.
We are hoping with this release to contribute to the community!
.
@github
... prbot is ridiculous insane.
CoPilot is inside your PRs... it's straight up suggests what code changes you should do, and gives you a PR that you can review, pull and see if it.. just worked.
#GitHubUniverse
🤯
Did you know? It's the second year running that
@k_cieslak
's work features in the
#GitHubUniverse
keynote. And it gets better every time! So glad to have him on the team at
#GitHubNext
.
It’s
#GitHubUniverse
today 🤩
The new project that I’ve been working on for last couple of months will be featured in the opening keynote… so don’t forget to tune in at (and don’t be late)
Every PR should come be covered by tests! GitHub Copilot checks whether tests are there, and if not it will suggest new tests for you. An experiment by
@GitHubNext
, we’d love to hear what you think!
Does code review improve code quality? If so, is it better to have every individual do more reviews, or to have more reviewers?
@TomBolton10
of
@SemmleInc
has the answers, with a careful analysis of the data on
@github
and
@lgtmhq
:
Plenty of cool stuff to do at
@SemmleInc
, with big practical impact! Come join the fun. Office locations in San Francisco, New York, Seattle, Copenhagen, Valencia and Oxford. Drop me a note to discuss the possibilities.
If you liked the talk by
@oegerikus
at
@ECOOPconf
#ECOOP18
: we're hiring! Join our team to build deep semantic code analysis for security research. . Internship applications welcome too!
Here
@shankuniyogi
explains the big picture of our security strategy at GitHub, and how
@semmle
is a natural part of that strategy. It's such a good fit!
So many amazing announcements today. GitHub Advanced Security is making it easier than ever to fix security vulnerabilities with our new code scanning autofix feature. Now when you introduce a vulnerability in your PR, we'll provide you a fix inline.
Total transparency is a necessity for securing open source. That's why everything on is visible to the whole community, and nothing is hidden:
@LGTM
@Semmle