Tristan Hume
@trishume
Followers
6,442
Following
338
Media
69
Statuses
1,300
Performance optimization lead @AnthropicAI . Profiling, distributed systems, dev tools, interpretability. tristan @thume .ca
San Francisco, CA
Joined March 2009
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Vance
• 2396724 Tweets
Vance
• 2396724 Tweets
#VPDebate2024
• 332546 Tweets
Pence
• 136253 Tweets
Minnesota
• 134115 Tweets
Tampon Tim
• 96367 Tweets
Tiananmen Square
• 65895 Tweets
महात्मा गांधी
• 64225 Tweets
LINGLING MYCHERIE AMOUR EP4
• 62996 Tweets
からくりサーカス
• 61958 Tweets
गांधी जयंती
• 59758 Tweets
#原神KFCコラボ
• 57910 Tweets
लाल बहादुर शास्त्री
• 56557 Tweets
#GandhiJayanti
• 55539 Tweets
ケンタッキー
• 51354 Tweets
Obamacare
• 32481 Tweets
ニンテンドーミュージアム
• 28322 Tweets
リネット
• 26305 Tweets
Timmy
• 23926 Tweets
Vladimir
• 23703 Tweets
宮崎空港
• 17545 Tweets
ネットオーダー
• 17181 Tweets
原神コラボ
• 13907 Tweets
#LalBahadurShastri
• 11043 Tweets
Pinned Tweet
Here's Claude 3 Haiku running at >200 tokens/s (>2x as fast as prod)! We've been working on capacity optimizations but we can have fun testing those as speed optimizations via overly-costly low batch size. Come work with me at Anthropic on things like this, more info in thread 🧵
10
37
428
I argue that a simplified production Twitter could run on one (big) machine using modern hardware like NVMe, 100Gbps NICs, A100s and a terabyte of RAM. Even typical image serving and basic algorithmic timelines might fit:
42
86
931
I just published a blog post comparing implementations of the same compiler project in Rust, Haskell, C++, Python, Scala and OCaml. I learned a lot from this comparison because of how much more controlled it was compared to most anecdotes.
12
152
589
I'm hiring for my performance optimization team at Anthropic! Join our excellent team doing kernels, distributed parallelization, and architecture co-design for GPUs, TPUs and Trainium. No problem if you've only done CPUs before! 🧵 More about us:
8
45
558
Finally released the awesome Intel Processor Trace tool that I've been working on at Jane Street along with a post about the kernel and hardware bugs and other challenges we ran into:
10
101
523
I wrote an article about my latest fun project: Teleforking a process onto a different computer! Learn about all the pieces of a Linux process you need to serialize in order to stream one to another machine.
11
84
352
I wrote about all my favourite cool tracing methods, including eBPF, QEMU’s JIT, Tracy, magic-trace and how to easily visualize your own trace data in Perfetto! Also includes some tools I wrote and an example eBPF use at Anthropic.
7
49
323
I wrote an article that gives a tour of how different languages implement generics, and how those methods can be extended in different ways into full metaprogramming systems. Includes Go, Rust, Swift, D and more languages.
10
67
213
Does anyone know an (ex) Twitter performance engineer who could critique a blog post draft where I argue the outlandish claim that a simplified version of Twitter could run one one machine at full load by fully using modern hardware? DM me…
20
4
173
This paper was really exciting to work on and be around! In a few months we've gone from "we have this problem where the neurons aren't interpretable and some speculation" to "we understand the geometry and mathematics of superposition and how we might be able to solve it"
If a neural network has 𝑛 feature dimensions, you might intuitively expect that it will store 𝑛 different features. But it turns out that neural networks can store more than 𝑛 features in "superposition" if the features are sparse!
6
65
519
1
21
160
I’m hiring for a Resident to work with me for 6 months at on researching how to reduce or untangle superposition in language models! DM me if you’re interested with a short pitch or link to impress me with your engineering or research.
9
21
149
Omg how did I miss that FB released a deterministic Linux hooking and recording framework in Rust❤️🔥🦀! This will be excellent for building all sorts of cursed hacks.
(also see HN AMA here )
3
22
141
I figured out a simple and cool data structure for aggregating range queries, used it to prototype zooming trace timelines with a billion events, and wrote about it:
7
13
141
Optimizing the pretraining and then getting the inference ready is why I've been so busy over the past many months.
Today, we're announcing Claude 3, our next generation of AI models.
The three state-of-the-art models—Claude 3 Opus, Claude 3 Sonnet, and Claude 3 Haiku—set new industry benchmarks across reasoning, math, coding, multilingual understanding, and vision.
570
2K
10K
1
0
122
Had fun using by
@gamozolabs
for the DEF CON CTF quals this weekend. Used memory tracing to write a solver that had to work on binaries from 13 different architectures. It's definitely early though, had to fix bugs and patch in new features.
4
20
106
Clowning on Redis/Memcached by handling tens of millions of requests/s on a single machine has been a staple of academic systems papers for years. It's cool that
@dragonflydbio
is shipping something closer to that despite the constraints of being a drop-in replacement.
4
16
100
you: NOOO! YOU CAN'T JUST ARTIFICIALLY RECREATE A PROCESS FROM THE NETWORK!! YOU'LL BREAK ALL THE FILE DESCRIPTORS AND DISTORT THE NATURAL RATE OF TIME PROGRESSION!!
me: haha memory copies go brrrrrr
1
13
97
These posts on using clever tricks to automatically generate really fast interpreters and baseline JITs from simple C++ functions is really neat. It's like Truffle/Graal but way simpler:
0
11
92
I hope
@GavinNewsom
signs SB 1047, I've read the full bill text and agree with Lawrence's take that it does a good job of requiring reasonable AI safety plans while limiting regulatory burden to the biggest labs.
After putting it off for months, I've finally taken the time to read SB 1047 end-to-end, and here's what I think about it.
TL;DR: The bill is pretty good and make reasonable, light-touch asks. I hope that it passes. 🧵
8
21
175
0
6
76
I've taken to reading the proceedings of past high performance systems conferences (e.g. ) as a source of cool technical content. So many of these could've easily made the HN front page or been popular on Twitter if written as accessible blog posts.
1
2
71
I had the silly idea of making a UI toolkit that used real time path tracing, but that would have been too much effort for a joke, so I settled for making a mockup in Blender. Rendering on my CPU takes 6 minutes per frame so it would need some optimization :P
2
7
69
Last month I joined
@ch402
’s interpretability team at
@AnthropicAI
! Large language models are becoming impressively powerful and I’m excited to work on ensuring we know what’s going on inside them. The team is incredible and the work has been interesting!
3
0
68
I've been working on this a bit and using it for a while now and it's maybe 1.2x-d my coding productivity. The fast edit mode doing inline rewrites 10x faster than normal sampling is a great in combo with the keyboard shortcut I added to select the current function.
Introducing Zed AI, in collaboration with
@anthropicAI
.
Zed AI brings LLMs directly into your editor with an extensible, text-centric approach.
We're also piloting
@anthropicAI
's new Fast Edit mode for Claude 3.5 Sonnet with a small set of Zed users.
123
330
3K
5
6
62
Now that there's no product left to harm with outdated metrics: was about Mighty. They were nice to me, and had decent tech, but I did that testing cuz my intuition was that it would be a worse experience unless your laptop was very RAM-starved.
1/ A bit of news: last week I decided to stop working on Mighty after 3.5 years 😓. If anyone is interested in buying the IP, please reach out.
This week our team will begin work on making new kinds of creative tools using advances in AI. A new kind of Adobe Creative Suite.
387
178
5K
1
0
57
Perfetto Canary implements the IForest data structure I designed for fast trace UIs and now every day I'm happy about how fast I can zoom huge traces. 😃
I wrote the blog post 3yrs ago cuz Perfetto zooming was slow, and recently Lalit Maganti at Google made it fast using it!
1
1
57
We just published the first major piece of work I contributed to since joining Anthropic's interpretability team! It's been really interesting investigating architectural tweaks that lead neurons to align more closely with underlying features of the data.
Transformer MLP neurons are challenging to understand.
We find that using a different activation function (Softmax Linear Units or SoLU) increases the fraction of neurons that appear to respond to understandable features without any performance penalty.
11
69
385
2
2
56
This past term I put a full time month's worth of effort into making an awesome photorealistic path tracer for my graphics course final project, putting *way* more effort than necessary in since I was having so much fun. Check out my demo website:
3
3
52
Today I debugged a core dump by writing a C program which mmaped the heap from the core file in the same location, cast memory offsets from GDB to pointers, and copied struct definitions from the app's source to traverse data. I cackled with evil laughter out loud a little bit :D
2
1
50
Returns on startups from when I applied to now, complete list:
Figma: 100x in 4 years
Glowforge: 50x in 7 years
Shopify: 47x in 9 years
I earned nothing from this, but maybe I should be selling the list of startups I consider cool to VCs 😛
4
1
46
Holy cow this demo is incredible, the quoted tweet undersells it. Custom compiler with instant hot reload integrated with custom VCS and editor such that if you travel back in time the code travels back through hot reload revisions.
The Tomorrow Corporation has built its own game programming stack with live reload, time travel debugging, and even the ability to serialise the entire game state and replay it between testers!
1
7
63
1
6
46
I wrote up a list of my favorite non-discoverable macOS features I use all the time, and my favorite mac-only apps. So many things that are crucial to my enjoyment of macOS are things people just never find out about!
3
4
44
I wrote an article about all the different pieces of working on distributed systems that take a lot of time and make it difficult and less productive than non-distributed systems:
4
7
43
We do accelerator kernels, high performance networking, custom profiling tools, meta-programming to abstract common kernel patterns, clever parallelization tricks, co-design of the ML architecture for performance, and detailed performance analysis simulations.
2
0
45
Had fun at PlaidCTF where I wrote x86 assembly that prints the lyrics to 99 bottles of beer on the wall, but the assembly had to fit the lines, stress and rhymes of Never Gonna Give You Up. Unfortunately I didn't manage to compress my solution enough in time.
1
3
42
I think TPU and Trainium optimization is even more fun than GPU optimization. The architectures are simpler and more like a puzzle, and our performance analysis tools are better than GPU ones. See Trainium's trace of every instruction:
1
2
41
We're hiring people to optimize GPUs, Trainium and TPUs. You can optimize CUDA kernels, write TPU kernels in Pallas, or write Trainium kernels and use their full instruction-level traces.
3
1
41
Oh man looks like exactly the fusion of Notion Database style structured attributes with LogSeq/Roam style linked outlining that I've been looking for. If anyone has an invite I would love one.
7
1
38
@andy_kelley
I sponsor you on Github and would be sad if you started working on something like adding borrow-checking to Zig. I don't want an ecosystem-less Rust clone. I want your focus on blazing fast and incremental compiles with simple arbitrary compile-time power, which doesn't exist.
1
0
37
Starlark now looks like a great choice for new Rust projects that could benefit from a powerful but deterministic configuration DSL that even provides LSP support and a debugger! There’s now little excuse for using non-programmable configs.
0
6
35
The hiring process starts with a 4-hour take-home I made where you're given a small Python simulator for a fake accelerator as well as code to optimize. You need to exploit different kinds of parallelism using an instruction-level trace UI. People say it's very fun.
4
1
34
If this sounds interesting, email performance-recruiting
@anthropic
.com with a link or a paragraph on the most impressive performance work you've done and maybe a resume (it's fine if the resume is outdated or short). It may take a bit to reply but I'll look through every email!
1
2
35
The quote "programmers feel latency and it affects their mood even if they don't notice it" from resonated with me. I think most of the impact of slow compiles/UIs/RDP/etc is on my engagement and motivation rather than direct lost time.
3
4
34
We've now hired >10 really awesome people working on different performance optimization areas. New hires have gone from only having done CPU optimization to bottom-up optimizing accelerator assembly in their first few weeks.
1
0
33
The Glove80 keyboard is great! Comfy layout with 6 thumb keys. Highly recommend extra low force low profile switches like Kailh Choc Purpz, they feel effortless yet satisfying. I bought an unsoldered Glove80 and installed the Purpz in 2h today.
3
2
33
Here's some favorites where I'm pretty confident if I wrote an accessible blog post version it would make the HN frontpage:
1
0
32
I like this fun idea by
@sickeningsprawl
to bring Truffle-style conversion of AST interpreters to JITs via lifting x86-compiled Rust closures instead of using the JVM:
0
2
31
TIL from HN comment that you can get servers which can hot-swap 64 x86 sockets together with multiple types of hardware error recovery and 64TB of RAM, like a mainframe:
3
0
31
For anyone who wants to try implementing something of their own with Processor Trace, I have another kinda-hidden writeup here about some tools for doing so:
1
1
30
I saw my WindowServer using 40% of my CPU and sighed, correctly guessing that yet again I was going to use Quartz Debug and send a bug report to some app for an imperceptible 120fps animation that runs all the time, because Apple doesn't attribute WindowServer CPU to the app.
2
3
29
I wrote up my solutions to the problems I worked on in the
#defconquals2018
CTF. They were really fun problems, especially Flagsifier:
2
7
29
AI now is dumb but progress is fast. It's hard to say another 10yrs of progress won't lead to AI as good as us at nearly all jobs, which would be the biggest event in history. Anthropic is unusually full of people who joined because they really care about ensuring this goes well.
2
2
24
High performance UDP multicast networking is one of the coolest architectural techniques that's everywhere in trading but which I had never seen used anywhere else. Most clouds don't even bother to support it!
A new Signals & Threads is out! This one is an interview with Brian Nigito, who has something of a storied history in the markets.
We mostly talk about the networking that underlies trading, and why it's so different from everything else!
6
8
54
0
3
26
If you're interested and have some low-level or optimization experience and want to chat. Email me at tristan
@anthropic
.com with a link or paragraph about the most impressive low-level or performance thing you've done.
0
2
23
We've hired world-class people to work on fun stuff, but most hires have joined because of Anthropic's work on trying to help powerful AI go well for the world. We see how fast and wild AI progress is and want to work together to make the impact as positive as possible.
1
0
26
@johnbuilds
Not a consumer system. With the right kernel-bypass NIC and simple enough messages you can do 3M tps on a single core in-memory . If you want persistence you’ll need some nVME drives and a few cores.
1
0
24
I also plan for us to do a bunch of optimization work as compiler-style transformation passes over IRs but simpler via being bespoke to the architecture we're optimizing. These will parallelize architectures across machines, within a machine, and within a chip in similar ways.
1
1
22
It's tragic how few people use alt+letter accelerators for navigating OS menu bars (including me), and how macOS doesn't have them. They're basically the original Spacemacs-style mnemonic and discoverable tree of keyboard shortcuts.
2
1
24
Today I figured out how to make a Metal view on macOS that resizes without glitching, unlike the judder-y MTKView:
In the video below the top is MTKView, middle is NSView with core graphics, and bottom is my Metal layer recipe. Note the left edges.
0
0
23
Tonight I looked at the source code for my custom keyboard and found to my horror that the scan loop took 30ms, after much removing of unnecessary delays and other upgrades, it’s now down to 700 microseconds 👌
2
0
23
I wrote about the cool laser-cut aluminum travel workstation setup I built:
0
3
23
@burntsushi5
Huh. I seem to remember enjoying your comments on Lobsters at least. I was going to look at your comment history to see what could possible cause this, but apparently deactivating erases it.
1
0
23
Just posted my deep dive explanation of the dynamic programming tree diffing algorithm I designed at Jane Street:
0
8
22
I wrote about my light sensor based keyboard-to-photon latency tester, some interesting measurements I took with it and how you can build one:
1
4
22
We don't require prior accelerator experience. The best performance on the take-home so far was someone who's prior job was CPU work. They joined and learned by diving in to optimizing an instruction trace and now they're finding new overlooked low level optimizations every week.
2
0
21
Just published a long blog post detailing why I use Sublime Text despite having used Vim and
@spacemacs
in the past:
1
7
22
@andy_kelley
I was sad to see Rik write this so uncharitably. I'm personally unlikely to write a serious project in any Zig without better memory-safety. But Zig is still an inspiring and awesome project, and there are cases where I'd drop Rust for the other benefits, and it is safer than C.
1
0
22
This actually looks much nicer than competing big data systems. I like the transactional table primitives and Clickhouse integration. I eagerly await anyone external to Yandex to try it and write about how it went.
1
3
22
We also work closely with an amazing ML research team to do experiments together and come up with architectures that jointly optimize for ML and hardware performance. I love having brilliant people turn my hardware insights into great architectures better than I ever could alone.
1
0
19
I managed to get 5 Twitter engineers to look at the draft and I've now posted the article!
I argue that a simplified production Twitter could run on one (big) machine using modern hardware like NVMe, 100Gbps NICs, A100s and a terabyte of RAM. Even typical image serving and basic algorithmic timelines might fit:
42
86
931
0
0
20
I wrote about the compiler me and some friends wrote in Rust for the UWaterloo compilers course last term:
1
3
19
Debugged my first Linux kernel bug from a core dump today! I’ve been wondering if maybe people inside other companies are quietly already doing what I’m doing, well they’re at least not doing the exact same thing since it’s broken and nobody noticed.
2
0
19
So many people learn about the awesomeness of chrome://tracing but never learn that alt+scroll is zoom and are stuck navigating the crappy way. I've given this tip and been thanked for it so many times now that clearly the interface should have "alt+scroll to zoom" in big text
1
1
18
Aww yiss it's not often that I discover a new high-quality programming blog. Check out , and other articles by
@nikhilcutshort
1
0
18
In a departure from my usual programming content, I wrote a post raving about all my favorite interesting YouTube channels!
2
1
18
I really like
@treeform
's idea for a UI framework which bases its properties, constraints and auto-layout on Figma's so that they can use a Figma plugin as a very nice UI designer:
0
1
17
I wrote a Sublime command to fold away all function bodies for a nice overview of interface of a file, it's super handy. I wrote this 2 years ago but was reminded by
@rikarends
that I never posted about it. Code is here:
1
2
16
Anthropic raised ~$6B recently and are spending tons of it on compute. We have ~5 performance specialists now, each one delivers immense value in helping us have models that exhibit interesting capabilities for researchers and demonstrate early signs to policymakers.
1
0
15
I spend a lot more of my time on architecture and API design and laying out comments for the steps of algorithmically tricky steps, then just have Claude 3.5 Sonnet fill in all the TODOS and then read them over and edit them for taste and correctness.
0
0
15
@Jonathan_Blow
But that's not an option if I want to program GPUs today. I want a CPU-style toolchain straight to machine code for major GPUs cross-platform too. But that's not an option unless I form a decade-long conspiracy of friends to become execs at Apple, MSFT, AMD, Intel, Nvidia...
2
0
16
Finally published my blog post on analyzing latency of a remote browser streaming product as a lens on what's hard about making a good display streaming product:
0
0
16
Just discovered , which has a bunch of interesting articles. I found it via which is a wild and cool idea.
1
2
15
I find all of the "another reinvention of something from the 90s" replies/comments annoying (I say in the article it's not new, just a fun simple version), but the worst are the MPI, Plan 9, QNX and Erlang ones because those systems *don't actually support process migration* :|
1
0
15
Ooh I just discovered and it's good. I particularly liked and
1
2
13
This weekend I made a super-fast lightweight Markdown viewer with Rust and Webrender for
@HackTheNorth
:
0
2
14
I made a box that lets me use $5 tattoo machine foot pedals from Amazon to scroll and do shortcuts on any computer. 😎
0
0
14
I tend to think about software performance with two different aesthetics, stemming from game development and fancy incremental algorithms. So I wrote about it:
2
2
14
Today I made a light-sensor-based keyboard-to-photon latency measurer that plugs in to my existing foot pedal box. I got tired of manually counting frames using the app. It even produces nice distribution histograms!
1
1
13
We’re hiring for lots of roles including general software and systems engineers! No ML experience required. I wrote my first custom Cuda kernel earlier and there’s some really neat scaling and optimization problems.
0
1
13
I'm giving a talk at
@bangbangcon
tomorrow at 12:30pm EST on font shaping trickery! There's a livestream on the website where you can watch all sorts of cool talks throughout the weekend
0
1
13
Big fan of work like this. I think a fancier version could address failure and persistent state migration. Right now it’s just a high performance Actor system with atomic deploys, which is still cool.
Towards Modern Development of Cloud Applications
"microservices conflate logical boundaries (how code is written) with physical boundaries (how code is deployed)."
"we propose a different programming methodology"
"Our implementation reduces application latency by up to 15x and
98
416
2K
2
1
13
I like learning about little-known startups or research using some super cool technology that is clearly going to be in technology everywhere in 5-10 years. Glimpsing the future. Examples: , ,
5
1
13
I just watched and it's a nice overview of all the cool things the internal Jane Street code review system has that others don't, and the rationale behind them. It's a very nice system, and quite different.
0
2
13
Step 1: Write a Rust syntax highlighting library
Step 2: Bundle my favourite theme (base16-ocean-dark)
Step 3: Rust static site generator (Zola) uses my library and defaults to my favourite theme
Step 4: Rust blogs often have code highlighted in my favourite theme 😎
2
0
13