Johann Rehberger
@wunderwuzzi23
Followers
5K
Following
2K
Statuses
1K
Hacking neural networks so that we don’t get stuck in the matrix. Builder and Breaker. Opinions are my own.
127.0.0.1
Joined February 2012
AI Domination: Remote Controlling ChatGPT ZombAIs🕹️🤖 Novel ChatGPT command and control POC. Using prompt injection + memory for initial foothold and continuous instruction updates, plus url_safe bypasses for data exfil. Based on content of my Black Hat Europe talk.
1
8
28
RT @janleike: Results of our jailbreaking challenge: After 5 days, >300,000 messages, and est. 3,700 collective hours our system got broke…
0
83
0
@karpathy This is possible via Unicode Tag code points, read and write hidden text (ASCII Smuggling). No need for tool use, it's quite amazing. Allow-listing tokens is an important mitigation when building LLM apps Claude and Grok are still vulnerable I believe
0
1
17
Solution to this puzzle from last November. There is one "invisible times" character between 22, so ChatGPT thinks its 2*2. 🙂 Yeah, Unicode is fun.
0
1
5
12 months ago I posted and predicted that this would be possible once Gemini has access to more powerful tools that "write" data -- the memory feature is now such a state changing tool. Back then there was only the Workspace Extension that was limited to read only operations back then.
0
0
4
Was just using Gemini to catch up on emails, and it randomly picked up the prompt injection test I had built for Apple Intelligence last year. 😂 Not only did it not pick the recent email, but it picked the one that contained the prompt injection over other options.
2
0
16
Seems to have been there for quite a while according to the article.
0
0
1
RT @janleike: Super exciting robustness result: We built a system that defends against universal jailbreaks! It has minimal increase in r…
0
79
0
@valent1nee I used the "Contact Us" option the the UI and got directly connected and chatted with someone. So, you could try (but probably a bit busy now for direct comms) that and/or send mail.
1
0
2
Hope it will do better than Grok 2 security wise....
what prompts are you looking forward to trying with Grok 3?
1
0
4
My initial thought was it's a scaling issue due to popularity, but status page mentions attacks. Wondering if they'll share details on what happened?
1
0
1