shayonj Profile Banner
Shayon Mukherjee Profile
Shayon Mukherjee

@shayonj

Followers
992
Following
28K
Statuses
3K

Everything, everywhere - always in progress. Do less, well. πŸƒβ€β™‚οΈπŸ‘¨πŸ½β€πŸ³πŸ’»

Cambridge, MA
Joined October 2009
Don't wanna be here? Send us removal request.
@shayonj
Shayon Mukherjee
2 days
0
0
1
@shayonj
Shayon Mukherjee
3 days
Heya! Thanks for reading. I don't have specific numbers, mostly because it's meant to be a high level doc (mentioned in the post). However, these are mostly references from my personal experience from managing 100+ PG clusters, where each cluster manages 10-12 k commits/s at peak with 15-30TB database size at peak, if that helps :) I must say, i totally forgot about connections when writing that post πŸ˜‚. Perhaps in the next one. I'd be curious to learn about any anecdotes from your experience too πŸ™πŸ½
2
0
2
@shayonj
Shayon Mukherjee
3 days
AGI will be achieved when it can root cause a segfault
@rakyll
Jaana Dogan γƒ€γƒŠ ドガン
3 days
AGI will be achieved when there are no flaky tests.
0
2
4
@shayonj
Shayon Mukherjee
3 days
RT @tines_hq: We’re thrilled to announce our Series C fundraise and company valuation of $1.125 billion. ✨ WATCH: CEO @eoinhinchy shares w…
0
6
0
@shayonj
Shayon Mukherjee
4 days
@thdxr PlanetScale is legit cracked
0
0
9
@shayonj
Shayon Mukherjee
9 days
@saikatc Stoked for you! All the best
0
0
2
@shayonj
Shayon Mukherjee
10 days
Using a proxy table for counter helps where the only updates that are done are for counters. Something like this (from an internal tech plan) Load testing showed better results, also ofc because reads are extremely fast now, while writes do take a bit of an hit. Be interesting to do a load test and compare the results between updating the single row vs spread across 100 rows/slots.
Tweet media one
0
0
1
@shayonj
Shayon Mukherjee
11 days
@rony358 Very cool! Thanks for sharing, I have bookmarked it a few times but then keep missing it :).
0
0
1
@shayonj
Shayon Mukherjee
11 days
RT @rony358: @shayonj We operate a large fleet, although we are not the largest. Architecture described here Sizes…
0
1
0
@shayonj
Shayon Mukherjee
11 days
@isamlambert That has been my sense too. Esp when it comes to QPS. Good examples of larger companies doing sharded setups fwiw.
0
0
1
@shayonj
Shayon Mukherjee
11 days
@arundsharma @filipecrosk @zerodhaonline @RobinhoodApp Very interesting , thanks for sharing
0
0
0
@shayonj
Shayon Mukherjee
27 days
Really cool and impactful work from the folks at @paradedb on how they moved their Tantivy-powered search from the file system to Postgres block storage. Really clear explanation of the challenges and solutions πŸš€ (must read)
@philippemnoel
Philippe NoΓ«l
28 days
1/11. ParadeDB is now integrated with Postgres block storage. As far as we know, no one has integrated a search and analytics engine with Postgres storage before. This is a big deal. Here's why we did it, how we did it, and why you should care. 🧡
0
1
6
@shayonj
Shayon Mukherjee
1 month
RT @sunbains: This makes sense from a practical perspective. People obsess over single digit latency like 0-60 mph car acceleration when…
0
3
0
@shayonj
Shayon Mukherjee
1 month
RT @ericfisher: Absolutely perfect flakes today ❄️ #Dendrites
Tweet media one
0
41
0
@shayonj
Shayon Mukherjee
1 month
@_mfridman Very nice indeed. A small footprint that takes in flags and falls back to env vars (a feature req if I may) is what I usually need 99% of the time :).
0
0
1
@shayonj
Shayon Mukherjee
1 month
@eatonphil Just these two for now. I found the basics and guides to ingredient sections extremely useful. Especially getting the β€œscience” right of what goes well with what, how to use existing ingredients for the resources required, building good mental models and so on.
Tweet media one
0
0
1
@shayonj
Shayon Mukherjee
1 month
I have been recently looking into LSM trees as I think it solves certain use cases very well. Stumbled upon this thread in Postgres ML - I would _love_ a PG compatible (fully) LSM solution if possible. And yes I did try to bake in rocksdb into PG, it "worked" but also failed hilariously :D.
@iavins
v
1 month
I would welcome an empirical study or survey paper comparing B-Trees and LSM Trees, focusing on performance, resource utilization, and code complexity
1
0
4