Yan Cui
@theburningmonk
Followers
23,030
Following
2,241
Media
1,516
Statuses
13,964
I help you master AWS and serverless | AWS Serverless Hero Newsletter 👉 YouTube ▶️
Amsterdam, The Netherlands
Joined February 2011
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Stop FakeNews On Sant RampalJi
• 407834 Tweets
ゼンゼロ
• 201578 Tweets
Independence Day
• 181924 Tweets
Happy 4th of July
• 154168 Tweets
ゼンレスゾーンゼロ
• 137421 Tweets
#あや姉さん一生ついて行きます
• 53692 Tweets
HANBIN
• 47235 Tweets
#ILAND2
• 40497 Tweets
インティマシーコーディネーター
• 37227 Tweets
Declaration of Independence
• 33664 Tweets
Ruffin
• 31219 Tweets
Happy Fourth of July
• 30692 Tweets
#VictoryParade
• 29998 Tweets
God Bless America
• 22750 Tweets
Hesitate
• 21715 Tweets
VLOG 3 WITH BUILD
• 21370 Tweets
izna
• 17709 Tweets
#バンドリ大発表会2024
• 17400 Tweets
GULF X MD
• 17344 Tweets
Happy Birthday America
• 16450 Tweets
アンジュルム
• 15886 Tweets
#ファフナー大祝福
• 12555 Tweets
SQUARE_ONE
• 12310 Tweets
Zafer İşareti
• 12293 Tweets
川村文乃
• 11987 Tweets
This is ridiculous, Netflix's monthly EC2 spend was $19M in 2020 (source: ) when EVERYONE was binging TV shows at home.
No way an SEO company would spend anywhere near that.
Did someone forget to tell them the E in EC2 stands for "elastic"?
53
77
896
It was a long time in the making, but it’s finally here! 🥳🎉🎉
65
120
868
AWS has 200+ services but you only need to learn these 10 to cover 90% of use cases and be very productive.
• 𝗦3: data storage
• 𝗗𝘆𝗻𝗮𝗺𝗼𝗗𝗕: scalable database with predictable performance at any scale
• 𝗟𝗮𝗺𝗯𝗱𝗮: serverless compute
• 𝗖𝗼𝗴𝗻𝗶𝘁𝗼: identity
26
126
861
At DAZN, we had over 2M concurrent viewers at peak and crazy spiky traffic (everyone logs in 30s before a match starts).
We had a similar principle, that any service on that critical path (of a user logging in and start watching a live event) has to run on containers, because of
19
154
836
The no. 1 question I get about
#serverless
is around testing - how should I test these cloud-hosted functions? Should I use local simulators? How do I run these in my CI/CD pipeline?
Here are my thoughts on this 🧵
23
138
601
Launched a new app for client 3 weeks ago on AppSync, 5,000 users later, 100,000+ API calls per day, 99% cache hit rate, 0 problems, and 90% of AWS cost is on AppSync cache (T2.Small). Easily one of the most complex things I've ever built (in a few weeks). Gotta love
#serverless
!
16
91
538
I have written ~800 technical articles/whitepapers in the last 10 years and they have been viewed millions of times.
I see a lot of folks making the same mistakes I did early on in my journey.
So here are some principles to help you get better at writing.
🧵
15
129
544
Ok, this is bonkers... S3 charges you for unauthorized requests 🤯 so someone can DDOS you by making unauthorized to your private bucket!
All it takes is for them to guess your bucket name 🤦♂️
19
109
479
For a lot of orgs, the easiest way to cut down on AWS spending is to give their devs access to billing.
Without a feedback loop, devs won't know the $ impact of their architectural decisions.
24
43
468
I have written over 800 technical articles in the last 14 years and they have been read millions of times.
I see a lot of folks making the same mistakes I did early on in my journey.
So here are some principles to help you get better at writing.
🧵
13
65
417
Earlier this week, Google, Cloudflare and AWS defended against the largest DDOS attack error, receiving 398, 201 and 155 million RPS respectively.
The technical details of these are fascinating! 📖👇
4
88
389
Going live with API Gateway and Lambda?
Make sure you go through this checklist first!
#serverless
#aws
#awslambda
#apigateway
6
72
375
Common API design patterns at Stripe.
Some really good advice here. And check out the earlier posts in the series as well, I really liked the post on Error messages.
2
67
358
How Stripe achieves 99.9999% uptime
0
73
355
This is big!
Payload-based filtering was one of the key reasons to choose EventBridge over SNS. This makes SNS a much more viable option in Event-Driven Architecture.
7
75
337
Event-Driven Architecture (EDA) is all the rage right now, and I love it too!
But it's not the best, or simplest solution for every problem. Sometimes, orchestration is the way to go.
Here's a guide to help you understand when to use which.
9
54
331
Serverless is great, but it's not a free lunch.
The "serviceful" mindset means you'll need to learn and use many services in your architecture. Having the right tool to help you along can make a world of difference. And here are 7 of my favourites.
8
66
302
"Designing APIs for humans: Error Messages"
Good article from Stripe on how to do error messages
* send the right HTTP status code
* be descriptive
* be helpful
* provide more info to help caller figure out what they did wrong
* be empathetic
2
51
274
Yay, Lambda now automatically detects infinite recursions and stops them! 🎊
This should put a stop to those billing horrors you hear about every now and then "I got a big AWS bill after my Lambda function got into an infinite loop!"
8
54
272
A common mistake in architecture diagrams is that they only depict data flows, not control flows.
@ghohpe
contextualizes the distinction and demonstrates how it helps you make better architectural decisions in this talk 👏
5
40
269
As the first AWS service, I almost forgot SQS is still stuck in XML land until I saw this announcement just now.
If you're using an official AWS SDK, there is no code change required on your part. Just upgrade the SDK to the specified version and it'll
9
37
252
Did you know that you might be 𝗱𝗼𝘂𝗯𝗹𝗲 𝗽𝗮𝘆𝗶𝗻𝗴 for requests handled by API Gateway? And it's all because of CORS.
It's easy to enable CORS on API Gateway, but you do end up having to pay for those preflight requests.
Because API Gateway charges you by request and data
11
29
248
When I was at DAZN, we had 𝗼𝘃𝗲𝗿 2,000,000 𝗰𝗼𝗻𝗰𝘂𝗿𝗿𝗲𝗻𝘁 𝘃𝗶𝗲𝘄𝗲𝗿𝘀 for big events like El Classico.
Almost everyone logs in just before kick-off, creating huge (100x) traffic spikes. Back then, Lambda couldn't handle spikes of this magnitude.
So we had a rule
4
26
244
Possibly the best answer to "What does a software architect do?" 🤘
2
35
240
re:Invent starts tomorrow, so let me round up the biggest
#serverless
related announcements from the last 2 weeks (I know, crazy!) and share a few thoughts on what they mean for you.
Mega 🧵If this gets 168 retweets then I'll add another batch!
#serverless
#aws
#reinvent
6
148
227
For years I mistook confidence for competence and thought I was the problem and I was missing obvious things. Only after I started asking "how" and "why" (and not let go until I get a good answer!) did I realise how often people bullshits their way through eng decisions... 🤦♂️
7
42
228
Free courses on Lambda and Step Functions!
I'm offering two of my most popular courses for free. Learn how to build serverless applications with AWS Lambda and Step Functions.
No strings attached.
Sign up here:
3
56
216
Nice, detailed post on how they designed a fully serverless notification system.
Would love to see more about the deduplication mechanisms. E.g. do they dedupe the frontend in case a message is sent via multiple channels
2
39
215
This is why you need composition over inheritance.
6
36
214
Holy shit, it finally happened!
Cognito has addressed a long-standing issue and lets you customize its access tokens 😍
You don't have to use id tokens anymore for custom attributes, yay!
6
33
213
this is why you need composition over inheritance... :s
#CompositionOverInheritance
http://t.co/aKRwUaZ0qo
10
273
205
Holy sh*t, it works!
Please, someone, make a Chrome plugin and automate this 🙏
8
16
205
Wow, Lambda just blew its burst concurrency limit wide open 🤯
This was a major blocker for some of my clients.
Instead of taking 14 mins to reach 10k peak account concurrency, now each function can hit 10k concurrency in < 2 min. EACH FUNCTION!
6
33
198
This is why you need composition over inheritance.
6
35
198
This is a biggie, you can now run a full API out of Lambda without API Gateway! 🚀🚀🚀
#aws
#serverless
4
46
189
X: (oversees a cluster of 100+ EC2 xlarge instances in dev that avg <5% util & costs $10k+/month)
(dev makes a mistake and a Lambda function went into an infinite loop and cost $50)
X: OMG, Lambda is terrible, we shouldn't use it ever!
perspectives people...
8
21
190
You can now use english to query CloudWatch logs and metrics. Very cool,
BUT... you still need to know your system!
The example below generates a valid query, but it doesn't work. It's a useful starting point but it doesn't know my system like I do.
7
35
187
The whole cloud repatriation movement didn't make sense to me.
But I get it now. Thanks to Wardley Maps.
The problem is not that these people don't "get" the cloud or that they are misguided. DHH is a smart man (as much as I don't agree with a lot of what he says).
The problem
12
38
185
How Discord stores billions of messages.
Interestingly, there's no mention of caching. Given the high read-to-write ratio (roughly 1:1, which is surprising), I wonder if they decided that it's not worth it 🤔
3
29
185
This is awesome!
Finally a good solution for the SQS over-polling problem that you can run into with Lambda. Previously the best workaround was to use FIFO queues and manage with group ids. Great to see an official solution for this 🙏
4
29
182
26
19
180
I've interviewed hundreds of engineers and been interviewed by dozens of companies, and certifications (or lack thereof) have never been a consideration in the outcome. Zilch.
The fact is, they're too broad & too shallow (in general) to be a useful way to gauge whether someone
11
24
182
Great question from my current cohort of students, paraphrased:
"Should you always use Step Functions to chain together a few Lambda functions? Are there patterns to simplify this? How about using SQS between the functions?"
Here are my thoughts 🧵
5
50
179
Now that Lambda Provisioned Concurrency is live, I can finally share my post on it - why it's important, what is it and how does it really work, and some rough edges to look out for for now (expect them to be fixed soon!)
#serverless
#aws
#awslambda
7
58
179
Here are 20 blogs that will help you get better at AWS & serverless.
Serverless Land
@jeremy_daly
's blog
@LeeJamesGilmore
's blog
Ready, Set, Cloud by
@AllenHeltonDev
:
8
42
178
Have you been looking forward to trying out Bedrock? Then you should give this article a read.
It shows you how to perform retrieval-augmented generation (RAG) with LangChain, Lambda and Bedrock, and build a "document chat" service (ie. a clone of Quivr)
3
39
176
Nearly 5000 copies of Serverless Architectures on AWS 2nd Edition have been sold 😮
Absolutely gobsmacked by the amount of support all of you have given us 🙏
If you haven't got your copy yet, Manning is doing a 50% off sale this week!
4
28
174
🤯 This is big!
6
27
168
It's been 12 months since I left full-time employment to go independent. It's been a crazy ride so far and I'm loving every minute of it. TL;DR of it is
1. more control over what I do and how much I work
2. higher earning potential
3. lots of learning
9
23
172
This is why you need composition over inheritance.
2
29
172
A client spent over $5,000 a month on a single Lambda function. That fact alone wouldn't be a problem if they were getting value from it and fully utilizing what they paid for.
The problem was that most of the CPU and memory were wasted.
The function performed a CPU-intensive
20
25
170
Going live with API Gateway and Lambda?
Not before you check out this checklist to make sure you're ready for it!
7
22
169
Some solid AWS cost-saving tips in this post. Not at all a sensationalist piece.
Some highlights/summaries. 🧵
3
35
167
Here's a regular reminder about hexagonal architecture, a powerful software architecture that makes your code more testable & portable.
In this video, you will learn what it is, why it's useful and when you should consider it in serverless architectures.
1
34
167
My 2020 in numbers:
✏️ 37 blog posts
📢 34 conference talks
📖 1 book
📽️ 2 video courses
👨🎓 2,475 students
🎙️ 43 episodes of Real-World Serverless
👥 5 webinars
💻 1236 contributions to 17 OSS projects
🤝 helped 17 clients adopt
#serverless
👨🏫 10 public workshops
see u in 2021!
16
1
163
Have you been stuck at "senior engineer" and not sure how to get to the next level? e.g. a principal or staff engineer
Here's my advice - start maximizing your impact, not your output.
Be warned, the principal engineer role is not for everyone.
4
38
162
A student asked me: "What AWS service should I use to collect and analyze click-stream events from the frontend?"
🧵
12
22
163
Have you been wondering how you should protect your API Gateway APIs - API Keys vs AWS_IAM vs Cognito vs Lambda authorizer vs Resource Policies?
Well, wonder no more.
I wrote a quick cheatsheet and ran through the options for you:
7
46
164
Are you spending too much on Lambda?
Here are some ways to save money on your Lambda bill.
🧵
#serverless
#awslambda
7
40
157
Don't wait for users to complain to find out there's a problem.
Here are the alerts you should have around your
#serverless
application.
It's not an exhaustive list, but a good starting point that's broadly applicable to everyone.
6
33
154
Aws dropping another bombshell of an update, silently, last night - you can now trigger lambda with managed Kafka. This is a big one for a lot of AWS customers, like Nordstrom, etc.
#serverless
#awslambda
4
61
156
I published a new CLI tool to help your
#serverless
dev. For now, you can analyze Lambda costs, list functions, tail async event sources, del
@goserverless
generated CF stack by name & replay SQS DLQ events
#aws
love to hear your feedback! cc/
@lumigo
4
52
155
I'm excited to tell you that my new course, Complete guide to AWS Step Functions, is finally published! You can browse the curriculum and purchase the course here: please RT to help me spread the word 🙏🙏
#serverless
#aws
7
96
150
Didn't think I'd ever say this, but looks like Cognito is finally getting some love and its future actually looks pretty bright.
30
3
152
This is a must-read if you're building event-driven architectures.
I'm generally all for local copies, but user data is the exception due to GDPR. A "right to erasure" request becomes very challenging to honour with all the scattered local copies.
4
28
151
Raising an event is easy, but including the right information in the event is a tricky balance. I like the distinction between domain and integration events described in this article.
3
30
153
I have spent quite a bit of time with AppSync on a few projects, and it's really grown on me, big time. So much of what's difficult with API GW comes out-of-the-box.
Here are 5 things that really stood out (a long form blog post to follow soonish)
#serverless
#aws
#graphql
1/
12
38
153
I'm starting a new weekly podcast "Real-World Serverless" where I discuss real-world use cases and challenges with
#serverless
technologies with engineers who are building stuff with them and get their stories from the trenches. Come give us a listen at !
7
45
139
AWS will start charging for public IPv4 addresses and it’s a good thing.
Great article by
@QuinnyPig
, explains the current situation with IPv4 addresses, the problem with AWS giving them away for free and what the pricing change means for all of us.
3
25
140
🚨 NEW POST TIME 🚨
"Choreography vs Orchestration in the land of Serverless"
I take a stab at another common question and explain why choreography (or event-driven) is not a good way to model biz workflows INSIDE a bounded context.
#aws
#serverless
10
42
135
Updating some materials for my next
#serverless
workshop in Aug.
Lesson - when you're doing cost projections, always take the targeted throughput into consideration, otherwise, you can be off by an order of magnitude...
4
30
137
Event-Driven Architecture (EDA) is all the rage right now, and I love it too!
But it's not the best, or simplest solution for every problem. Sometimes, orchestration is the way to go.
Here's a guide to help you understand when to use which.
0
22
137
AWS Secrets Manager is turning 5 years old next month.
It's great at what it does, but I still find myself using SSM Parameter Store most of the time, here's why.
"The Old Faithful: Why SSM Parameter Store still reigns over Secrets Manager"
12
30
135
Lambda now emits JSON structured logs for system logs as well as YOUR stdout messages. Here are the caveats you need to know before using it.
1. The sys logs use "time" for its timestamps, whereas application logs use "timestamp", so when you process log messages, make sure you
6
19
136
my 2019 in numbers:
68 posts
1.5m views
1.3k newsletter subscribers
2.2k students to video courses
31 conference talks in 17 cities
6 podcasts
4 webinars
4 chalk talks @ re:invent
9 SAR apps published
1792 contributions to 16 OSS
helped 23 clients in 11 cities
been a busy year!
11
0
136
SNS & EventBridge share many capabilities and can be used interchangeably in many situations.
So, how do you know when to use which? 🧵
#aws
#serverless
5
11
136
"
#serverless
cost is unpredictable" in one diagram ;-)
I think the big Q is whether you want to optimize for efficiency or make your budget more predictable. Both are valid - e.g. car makers optimize for unit cost; hedge funds pay huge amounts to make earning more predictable
3
47
132
My advice to junior developers
4
39
134
wow, THIS. IS. HUGE!! 🤯
I need to check this out myself, but at first glance, this should make importing existing resources so much easier and requires no manual actions
6
25
133
Student: Is it possible to implement WebSockets with serverless?
Me: Yes! In fact, here are 4 ways you can do it.
5
12
134
Just because you're writing tests doesn't mean you're doing TDD.
Just because Lambda invocations always have an event doesn't mean you're building an event-driven architecture.
Thanks to
@alexbdebrie
for opening this can of worms!
2
16
130
huh, API Gateway now lets you raise the 29s timeout limit, when did that happen?! 🤔
They should have made more noise about this, as it's a common problem people run into.
11
23
132
Here are all my posts on
#Serverless
and
#AWSLambda
covering everything from tips & tricks, ops, migration strategies, design patterns, security, findev and more
5
53
132
This is the best news I've heard in a while. Step Function has built-in support for re-starting a failed execution from the point of failure.
Tons of people can simplify their architecture by deleting their workarounds today.
👏👏👏
2
20
132
AppSync ➕ Lambda ➕ DynamoDB is a SUPER productive stack.
How productive you ask?
I once built a new social network for a client in just 4️⃣ weeks, working as the one-man backend team 💪
Here's how we did it. 👇
8
28
129
The best news I've heard all year. I've been waiting for this for a long time and so glad it's finally here!
Now you can write AppSync resolvers using javascript. 🎉🎉🎉
8
28
129
Events and Commands are often used side-by-side in event-driven architectures.
This is why the difference is important, and why you should use different services as the bus 🧵
#aws
#serverless
4
14
127