Maggie Johnson-Pint
@maggiepint
Followers
5,717
Following
1,200
Media
1,476
Statuses
19,032
Dog Person. DateTime weirdo. These days I work on planes. Forever ❤️JS. She/her @maggie .bsky.social @maggiepint @macaw .social
Woodinville, WA
Joined July 2014
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
BATALKAN BUKAN TUNDA
• 607333 Tweets
Polisi
• 594175 Tweets
FOTFOT NEW SINGLE
• 369614 Tweets
Mulyono
• 350145 Tweets
#SB19
• 343024 Tweets
#NHK_MUSICEXPO
• 334709 Tweets
Oprah
• 304399 Tweets
Gus Walz
• 193434 Tweets
El TSJ
• 143377 Tweets
Barron
• 52869 Tweets
Mikel Merino
• 47462 Tweets
Golpe de Estado
• 41016 Tweets
A'TIN ANG PANALO
• 40701 Tweets
Tribunal Supremo de Justicia
• 40013 Tweets
#BackToBackBillboardFanArmy
• 37484 Tweets
Eagles
• 37137 Tweets
Yale
• 32933 Tweets
#MAPATHEFIRSTTAKE
• 27792 Tweets
Pretender
• 25179 Tweets
Commanders
• 23402 Tweets
ちいかわ寿司
• 21804 Tweets
Dotson
• 18778 Tweets
#النصر_الرايد
• 18403 Tweets
Ann Coulter
• 12681 Tweets
チッケム
• 12454 Tweets
揖保乃糸
• 11632 Tweets
Milito
• 11508 Tweets
This is weird to say out loud, but I actually am kinda an expert in rate limiting, so I'm gonna explain some stuff.
About half of incidents in large-scale production systems involve having more requests than you can serve. There are two categories of this kind of incident:
160
2K
10K
1. Top-Down overload or "Reddit Hug of Death": This is what Bluesky experienced today - suddenly there was a HUGE demand surge and the servers just *couldn't* for a while. This also happens after superbowl ads or when pop stars announce tours or during DDOS attacks.
13
88
2K
2. Bottom-up: This is the less obvious and more common scenario, when something inside the system fails, that makes the system unable to serve normal load.
If you lose a redis cache and everything is reading to DB, you will drastically reduce your ability to serve requests.
6
71
2K
I don't know what happened at Twitter today, but I don't think Elon woke up and decided to shut it all down - my bet is some 'bottom up' problem (but not necessarily the DDOSd yourself problem everyone is tweeting about - that could be an effect of getting limited, not the cause)
6
82
2K
Anyways, hope this was informative to someone somewhere because it took a while to write 😂.
33
19
2K
The best rate limiters are 'adaptive', and can change rate limits based on system stress, priority of requests, and other things.
Twitter has a really good one because they had a really exceptional infra team until a year ago.
4
72
1K
Similarly, if a database replica, cloud region, or cluster goes down, you will be in a really tough spot for serving normal workload.
And of course if a developer on one service writes code that suddenly slams another service, that's "DDOSing Yourself" and is also bottom-up.
2
62
1K
My husband quit tech and ran a home improvement business for about a year. He was actually pretty good at it, had more business than he could take.
He went back to tech. Turns out it's unending 12 hour days and body pain for 1/3 the money.
For the farmer folks.
22
76
1K
@ask_aubry
He's gonna have a BIG surprise when he finds out the courts in pretty much all states won't let you take your kids out of state in divorce situations for basically any reason besides physical abuse.
3
3
1K
Even if they don't crash, requests stack up waiting for completion - this is called 'backup' - which is what causes the slowness in the requests that do work.
Backups have this bad effect of causing users to refresh the page, causing more requests and... more backups.
8
47
1K
In these scenarios, the rate limiter is the only thing standing between you and death - because of course if computers get hit with more requests than they can deal with eventually they OOM and crash.
1
51
1K
A lot of things you think are best practice are actually just your opinion.
16
181
1K
My hypothesis - Twitter lost a big part of a critical back end system - maybe they stopped paying their GCP bill, maybe they lost a critical cache and everything was reading other data, I truly do not know.
4
99
1K
Another: "I'm a product developer - why do I care about an infra problem?"
1. if you handle this in code you can do something other than give your users 'error'
2. If you handle this in client code, you can save the entire infrastructure by never sending. Literal hero shit.
3
47
1K
I was the first woman to present a proposal in person to TC39 (js lang committee w/30 year history). And I keep going.
#WiTBragDay
10
159
1K
What is the rate limiter tho?
At the simplest level, a rate limiter is a program that says "This computer can only do x requests per second" and stops all the others with "429 too many requests".
1
50
1K
In CS terms, this is implemented with a 'leaky bucket' algorithm - but that is not important unless you are making one.
Most good rate limiters can get pretty fancy and split the quotas by things like customer, customer plan (verified in twitter for instance), or feature.
1
32
1K
If you use it between all processes, then you can prevent one system from overloading another system, preventing all kinds of cascading failure scenarios.
When you do this, you also implement a key pattern: exponential backoff.
4
32
1K
Now, a lot of people only think of the rate limiter as something that goes at the 'front' of the infra to prevent the top-down kind of problem, but in fact advanced infra teams (including Twitter in the good times I'm sure) routinely use them *between all processes*
3
36
1K
Instead, you do an "exponential backoff" - first you retry in 1 second, then wait 2 seconds, then 4 seconds, then 8, 16, 32, 64 and so on (I used base 2 there but however you like)
This gives the servers a 'breather' if something really bad is going on - instead of slamming
5
34
1K
Then of course Elon gonna Elon so it's intentional 🤷♀️.
But really when you have a major outage you are intentionally going to do what Twitter engineering was doing, and have your requests come back in gradually to ensure you don't overload - their smart rate limiter can do that.
3
42
1K
At this point, their probably very good adaptive rate limiter said 'ohshit' and brought the number of requests WAY WAY down throughout the system.
The infinite loop screenshot floating around? Front end code sees the 429 and retries, but without exponential backoff.
4
37
950
Announced today that Moment will make no further major change or features. Closed a lot of issues with a link to our guide pointing people to other libraries.
Know what people did?
Reply to the issues with "Fine! I'm quitting for date-fns since you don't fix issues"
60
98
920
One common question I get is 'why not just autoscale?'
1. Not everything easily can
2. Autoscale is *expensive* to handle problems that only happen a couple minutes a day or year
3. It takes a few minutes, this fills the gap
4. Sometimes even the cloud runs out
7
31
899
If you find yourself configuring a rate limit on a daily basis that changes globally at midnight UTC ask yourself why because you are gonna get HUGE traffic spikes at that time and honestly you could be running way cheaper if you smeared the reset to the users local time.
13
44
876
When a program sees a rate limiting error (a 429 status), it's a signal that the request can potentially be retried after a bit and might succeed.
It seems logical to retry in a loop - but what if stuff is *Really* broken? A loop firing every second in every client is ‼️‼️
4
27
851
Now, all this context brings us back to today.
BEGIN PURE SPECULATION, I DO NOT KNOW!
2
27
830
It has been 692 days since I quit Microsoft and went to Stripe and I am STILL deeply angry about how inferior Google Slides is compared to PowerPoint.
I will never get over this change. Ever.
47
15
826
Spoke to an ex-Twitter SRE today who added an interesting tidbit about the rate limiting incident. He said "Even seeing the rate limiting message means multiple other things were broken".
I'm not surprised by this, because Twitter is a product with exceptional design.
3
86
674
Getting rate limited again. Assume we went over midnight UTC and this madness restarted.
Good grief.
3
20
667
Here are my financial interests in this topic. 😀
We are running a private beta - dm if interested.
If you didn't know - Stanza is a really flexible rate limiter. It lets you prioritize traffic, and throttle at the client with a few clicks.
Good rate limiting. It's important. We make a whole product for it.
1
12
97
3
28
647
So, does anyone else find the hating on front end intersectional with feminist, class and maybe race issues?
Front end devs are more often non traditional, came from design, were just making a personal website and then went deeper.
It feels like just another way to gatekeep.
28
34
473
Top down overload: "covid is here, everyone buy TP"
Bottom up overload: "can't staff the registers come back later"
12
25
442
The more I work in tech, the more I realize that a lot of people who are very visible are not seasoned tech pros. They're professional influencers.
It is something we should talk more about, because it can cause harm.
PSA: There's a lot of misinformation out there in these internets. Don't take someones tech opinion (including mine) as truth. Do the research, check credentials of your fav social media tech influencer. Have they actually worked in the space or have they just read about it?
1
21
110
16
74
411
We were all pretty heavy hearted with this. The library is an old friend.
And that's the response people give I guess.
The open source story needs work.
Software culture needs work.
8
18
371
@mxmultiversal
My stepson drew a bisexual flag, put it on his wall, immediately came and got me and showed me the flag. I said "are you telling me you are bisexual?" He said "yep. Figured it out yesterday from that book you gave me."
(It was Last Hearld Mage)
That was it.
10
6
352
Nerds. Nerds. Nerds. Nerds who make date libraries.
Open source is funny.
8
2
349
Accepted a job offer today. Super excited about it.
Get this, I'm going to work on tech for designing airplanes!
After seven years in dev tools, it felt like time to try another thing. This feels ultra-nerd in a good way.
29
5
345
@bertil_hatt
I've only really done this specifically for a year, though I've worked in general infra tools longer.
All my knowledge came from
@lauralifts
and
@equalize
who rather patiently explain to their stupid front end developer / product manager on a daily basis.
6
5
326
I started with TC39 just over a year ago. The first day that I went, I was completely terrified. I had been told that the committee was aggressive, could be hostile, and has no understanding of the needs of regular devs.
7
103
318
@AITA_reddit
ESH. Like, the kid certainly actually WAS being crappy, and the dad was being unhelpful. But the way to deal with it would have been to close the island and set a boundary with the brother - not cuss out a child.
2
0
301
Has anybody noticed that we are marching further and further into a liberal purity culture?
"I went on vacation" - "KILL THE COVID SINNER!!!!!"
"I work for CoinBase now" - "EXCOMMUNICATE THE PLANET KILLER AND GRIFTER!!!!!"
This stuff is getting ridiculous.
20
17
307
People acting like 250k should be enough for everyone and nobody should keep that much money in one bank haven't mentally processed that 2 week payroll for 100 folks making average 70k (school? small hospital? medium non-profit?) already exceeds that.
18
31
267
@Tore_Aurstad
@ashleymcnamara
@girlsnamewillis
Check out this awesome cake from my dungeons and dragons wedding! Sorry you hate happiness.
18
5
260
Are SREs just the group of people who experience the most outrage at the fundamental wrongness of stuff being broken?
15
19
260
Since this seems to be leaving tech Twitter and breaking into the real world, if you don't code at all this summary might help.
@ssotwelltx
You can kinda think of it as lines of people at the grocery store getting longer and longer (backup) so they switch to only letting 50 people in the grocery store at a time and you just call back every once in a while until they say it's your turn... if that helps at all.
3
16
151
2
20
263
First day at Stripe today!
Excited to try something new!
A little disappointed I have to start with the common cold but at least we all work from home so I won't spread it to anyone.
17
2
253
For those who don't know - I married my fellow maintainer Matt.
Open source can be really cool sometimes. 😍
3
5
255
@AITA_reddit
Must add, I have a similarly aged child, with similar interests, and I would be SO MAD if I found out he was being this disrespectful - in the online world or the real one. So like, seriously shame on this dude's brother, he is biggest TA IMO. But they're all TA.
3
0
195
@EricaJoy
Target keeps showing me (an large white lady) swimsuit ads with large black women in them and let me just say between that and this I'm really starting to get behind this company.
1
5
191
Gotta add - Tim, Iskren, Isaac, Kunal, Matt, and Lucas - You have been exceptional partners on this journey. And I've only ever met two of you IRL!
That part of OSS is amazing.
3
0
167
I suppose first thing in the morning is as good a time as any for this.
Today is my last day at Microsoft. It's been an awesome and life changing five years during which I've made many friends and grown a ton as a professional and a person. I will miss everyone a ton.
17
5
174
My manager left Microsoft, and I am temporarily doing his job, and his job has a lot of meetings, as it turns out.
9
1
169
@cassidoo
I'm pretty sure 80% of the reason I'm a PM is that I got sick of this cycle and was like "fine I will do your girl job".
I'm at peace with it now but I can't deny it's what happened.
6
8
163
@iandouglas736
I would reply with something like "I remember this was a hard requirement when we worked together at xyz co in 20xx. Fantastic to see leaders evolving the culture!"
He will know. He will know.
2
1
161
This re-org happened a couple weeks ago, but I'm just sitting around still being a bit amazed that my reporting chain goes me -> Lori -> Mandy -> Amanda -> Julia.
See the trend?
If you're into women led orgs, we're hiring.
8
14
156
@mxmultiversal
He also has tons of emerging identities in his friend group. It's a cool time.
1
2
157
@bocxtop
Some doctor may think this is a bad idea I don't know and I don't care it works amazing. There is a clear drug interaction and I want it.
2
1
153
Behold!
Can I buy a craiglist claw machine for $300 and put it in my basement and fill it with random things?
I want to. I want it.
2
0
15
7
2
154
He functioned like a tech lead - doing work and finding less experienced laborers and training them as they worked.
Because we were direct and not through a remodeler, we got up close with how a lot of this worked, and I'm telling ya - it's easily as complex as software.
3
2
154
10 years as a tech professional and I make my husband fix printers.
It's not that I can't.
I just hate them.
25
5
148
@ask_aubry
The court will take quality of life arguments sometimes but it has to be an extreme difference like 'my kid will be food and housing insecure if we don't go live with family' and that is not this.
0
0
146
@ssotwelltx
You can kinda think of it as lines of people at the grocery store getting longer and longer (backup) so they switch to only letting 50 people in the grocery store at a time and you just call back every once in a while until they say it's your turn... if that helps at all.
3
16
151
In tech, we act like the trades are clearly simple.
Hamed, the guy who did our bathrooms, was not a remodeling company. He was a crew lead - they actually are usually subcontracted by the remodeling company you hire and bounce from job to job. We were lucky to hire him direct.
3
2
150
@Karnythia
That these women miss that everyone always worked boggles the mind.
Even wealthy women, while not precisely working, were working very hard to maintain societal position. Pride and Prejudice has been available for reading since 1813 if you need more info.
4
5
144
Learning a new corporate culture is stressful AF.
To be clear, there's nothing wrong at Stripe. The norms are just different, and figuring out if one met the norms or just royally messed up is hard.
5
6
139
In my career as an engineer I got the feedback that I needed to be a PM because I was so focused on driving initiatives.
In my career as a PM I have been told I am to tech focused and ask too many detailed technical questions.
😐
11
2
137
I don't think people fully understand - In powerpoint you can crap whatever shit you want to onto a slide and there is a button called 'design ideas' that just... makes it look good. Automatically.
Why does nobody else have this?
I just tried Canva - not there.
6
1
133
Minor career update: still with Microsoft, but headed over to the Azure SDK team (under
@amandaksilver
) to be a program manager working on GitHub management, CI/CD, and the general community story. Really excited to be back in the warm embrace of my open source people!
8
5
132
I just realized that the two tech things I know way more about than most people are DateTime APIs and Rate Limiters.
So weird, huge nerd, can't even express the odd over here.
9
2
127
JavaScript community, I just...
There's a pandemic and thousands are dying and you have to hot take about excessive library consumption by volunteers making websites to try to help?
Stop.
Just stop.
3
14
127
Matt is basically a really good hobbyist BTW. He's great for dog doors, small drywall patches, sheds, fence repair.
We paid a guy with 20 years experience leading remodeling crews to do our bathrooms, and Matt would be the first person to tell you how much better he was.
1
2
128
Yesterday I met a Microsoft colleague who started in tech one year ago after 13 years of full time mom-ing. She is going to start contributing to open source, which she said can't be harder that raising teenagers.
#womenintech
#fierce
1
8
119
Since it keeps being brought up in the context of Moment this weekend: I remain extremely skeptical that sponsorship provides a long term solution to OSS sustainability. I have kids and a home. I need employment for health insurance and to make sure the mortgage is paid.
8
18
120
1
0
118
I'm out of Microsoft so I guess I can actually say this.
Why is it that the .NET community immediately escalates to the highest level of offense and has to defend the honor of this or that? Especially when a woman is talking.
8
6
116
@GovInslee
@SandraPersing
We need to close bars and restaurants. I know this is devastating to businesses, but the alternative is worse.
12
4
112
@RottenInDenmark
This is more like if the pilot guy had a private pilot license and made side money doing charters. Which would, I must add, be COMPLETELY DIFFERENT than what it was.
0
0
109
Graceful degradation - changing the experience in subtle and user-friendly ways that preserve critical functionality - is very common in high performing teams. And instead of achieving 9s, it achieves what matters.
Users can get their important stuff done.
1
7
110
That tile and window work is PRO my friends. Flawless.
2
0
113
I married my best friend yesterday. Love you so much
@mj1856
! King and Queen of Woodindog forever!
14
3
109
@SwiftOnSecurity
As the maintainer of a popular datetime lib (
@momentjs
), can tell you most time zones unnecessary political construct
3
8
105
Open source funding - long unpopular opinion:
13
85
102
Okay, less than 24 hours into BlueSky and all I can say is:
Twitters fixed.
Get your invite code so we can all leave these ruins.
6
2
99
When we brought in a construction plumber (contact of Hamed's) for a major plumbing reconfiguration, they were all watching it the way we watch Twitch streamers write code - to admire the work.
1
1
104
I'm glad to be a progressive in a time where we think so deeply about ethics. It's a great thing.
But we also live in an imperfect time and we are all going to have to make imperfect choices. And we will all have different points of view on them. At some point give grace.
2
7
103
Congratulations to my new CVP
@amandaksilver
- for the first time ever I woke up, read a re-org mail, and was in a good mood for the rest of the day because of it. Amazing leader, glad to see it recognized.
5
4
101
Sometimes, when I can't work the XBox UI, I say real loud "I am a senior software engineer at Microsoft!"
11
8
97
If you didn't know - Stanza is a really flexible rate limiter. It lets you prioritize traffic, and throttle at the client with a few clicks.
Good rate limiting. It's important. We make a whole product for it.
1
12
97
It's my birthday so I get an afternoon donut and wine.
I also had a morning donut.
23
1
98
Got legal approval today! I'm officially building *open source* crisis management tools for
@Azure
. DM me if you want to join the team!
6
27
97
Gonna have a really unpopular opinion, but this "my wife is great" thing feels like dudes talking about us like accessories to their tech influencer cereers.
30
3
95
That meta rename went really well we are all in the metaverse right now.
And we all use a wide variety of Alphabet products.
8
10
93