⭐️ EventCatalog v2 is now live ⭐️
EventCatalog is an open source project to help you bring discoverability to your event-driven architectures.
EventCatalog v2 includes new features:
⭐️ A refreshed UI/UX
⭐️ Documenting domains, services and messages
⭐️ New visualiser for your
Messages vs events, what's the difference within your event architectures?
Here are some of my quick notes + visuals to help (myself) and might help you 👇
Whats the difference between queues, streams and pub/sub messaging patterns?
New visual here to help you dive deeper, with resources to learn the difference between them.
Queues
1. Messages are put onto a queue
2. Consumers pull and process them
3. Multiple consumers can help
Event-driven architecture with domain-driven design, why do they work so well together?
Here are some of my thoughts on it with a list of extra resources for you all 👇
Domain-driven design and event-driven architectures go well together.
Defining clear boundaries/language in your system (bounded context) and use events to communicate can help scale, remain decoupled, and define models.
Bite sized notes to learn more 👇
Welcome to Monday 👋
To kick things off, here are some high level thoughts on why messaging/events can help you create scalable, reliable and fault tolerant solutions.
Visual and resources 👇
As you might be aware I'm a huge fan of Amazon EventBridge 😅♥️
Over the past year, I have created some open source projects to help the community extend EventBridge functionality and write tools for development.
Here is a list of tools I have written to help you all 👇🧵
Event-driven architecture provides decoupling, but we can remain coupled through payloads (often overlooked).
The "Message Translator" pattern can help.
Coupled with DDD context mapping can lead to some interesting thoughts.
Visual and resources 👇
What is Event Sourcing?
New visual to help you understand Event Sourcing, some benefits and resources to dive deeper.
Store events in append-only log
1. Events are immutable and stored in order
2. Read/write and discouraged to update/delete
3. Changes to events are new events
Into event-driven architectures? Want to learn more?
Last week I started to build EDA Visuals and now have 20 bite sizes visuals to help you learn.
I have now compressed these into PDF format for you all to download for free 🚀
Hope it helps 👇
What happens if the same message/event get triggered to your downstream consumers? Are you ready, any side effects?
I remember this tripping me up 🙈having side effects in consumers, I learnt the hard way..
Visual to help you understand and resources 👇
When building event-driven applications you will may come across "eventual consistency".
What does it mean? Why is it important to understand? How can it impact your users (if not careful).
Visual and resources 👇
What are events in event-driven architecture?
Events may sounds simple... but dive deeper and there is much more to learn💡📖
In this new bite sized visual I help you learn more and give resources to dive deeper 🚀
Immutable Facts
1. Events are immutable
2. Cannot be changed
Fundamentals of event-driven architecture
Here are some thoughts I have been diving deep into this year, with things I learn about EDA I wanted to group them into three areas I think can help when you build EDA applications.
Identify and design
Like most things I think you
Sometimes when you build event driven applications, you may need to handle large payloads.
Some brokers may have limits, a way around this is to use the claim check pattern.
Store the data and retrieve the data downstream 🔑
More info + resources👇
Understanding eventual consistency
1. Distributed State
When we build distributed systems, there are times that state is distributed across our architecture.This means data across your architecture in theory will be eventually consistent and at times the state will be
Building event-driven applications?
It can be important to know the different types of events you can raise and the tradeoffs, something I learnt after building EDA apps... wish I knew sooner 😅
I spent 2022 diving into event-driven architectures, looking at event storming, schemas, DDD and more...
Conclusion: Event identification & design is important for EDA.
Here are more thoughts on what I capture as "Event first thinking" 👇
Here are some thoughts of a few things that are good and hard about event-driven architectures.
EDA can bring many benefits, but like most things there are things to consider and deal with and tradeoffs to make.
Sync vs Async models.
Our architecture will consist of sync and async communication methods, EDA leaning towards async processing.
Here are quick notes and visual to help understand the two 👇
I've been thinking recently what makes our architecture event-driven?...
It's a mixture of event/message patterns.
Queues, Pub/Sub, change data capture, streaming, process managers.... all sorts.
My bite sizes thoughts 👇
Great thing about the material on (new home soon), is they are just markdown files.
In theory I could export them and create cheat sheets for you all, or even a poster for your wall 😂....
What's the difference between batching and event streams? When should you use one over the other?
Events are super important in our event-driven architectures, so understanding these fundamentals can help.
Visual and resources 👇
I just launched !
Easily find AWS icons.
✨ UI to find AWS Icons
✨ Copy and Paste with one click
✨ Tag-based search
✨ Dark mode 😎
✨ Preferences sync to localStorage
If you find it useful, a retweet would be awesome! ❤️🌍
With event-driven architecture sometimes your downstream consumer needs more information than what is in the event payload.
They can go back and request it.... 🤔
Or you can enrich the event before it reaches downstream consumers 👇
Event-driven architectures (EDA) can provide agility, resilient and decoupled solutions.
Domain driven design (DDD) can assist us designing and creating software that delivers value.
IMO this is why EDA + DDD are a match made in heaven ❤️
What happens when events fail to reach or get processed by downstream consumers?
Dead letter queues, re-drive, outbox pattern, archive/store, or just drop the events?
Visuals and resources 👇
When building event-driven applications, no doubt you will have some point-to-point messaging in there.
Point-to-point messaging can help your applications scale, provide buffers and help create async patterns.
Visual and resources to help 👇
"Organizations designing systems will create designs mirroring their communication structures." - Conway's law
I think there is a huge overlap between Conway's law, domain driven design and event-driven architecture 🤔
My thoughts and resources 👇
If you are into Event Architecture and want an Open Source tool to help:
🔎 Discover Events
📄 Document Events
📊 Visualise Consumers/Producers
🧑💻 Schema + Code Examples
And much more...
I'm working on something that you might be interested in 👇🧵
When building message/event based solutions you may want to take a message or event and split it into many different ones.
We can use this to split large messages/events into smaller ones for downstream consumers.
Visual, resources and example 👇
Interested in building event-driven architectures?
You will need to understand the behaviour of your system, identify events, commands and aggregates.
How? EventStorming can help.
Learn more and extra resources 👇
I have some news to share with you all, I have decided to leave AWS and take a leap into more open source work ❤️🚀
The past 2 years has been an absolute blast at AWS, working with some fantastic friends. The Serverless DA team is one of the best teams I have ever worked with.
This is why I love event-driven architectures, they can enable us technically and can have an impact your business.
Events to identify opportunity/innovation, real-time reports, understand state through events, and much more...
Visual and resources 👇
Internal events, external events, domain events, integration events... what does it all mean?
Using events to communicate within a domain and events between domains can be (and should be) treated differently.
Here are some of my thoughts 👇
Here are some common things that catch you out as you scale your event-driven architectures...
After speaking with many people building EDA applications, so many of these issues arise.
Here is the visual with resources to help you avoid these issues 👇
I just launched EventCatalog!
An open-source tool to help you document your Event Architectures ⭐️
🔎 Discover Events
📃 Document Events
📊 Visualise
🧑💻 Schema + Code Examples
🤩 And much more...
If you find it useful, a retweet would be awesome! ♥️🌎
Thinking about building event-driven architectures/applications?
Here are my thoughts on what you should consider when building EDA applications.
1. Lack of idempotency handling
Consumers may process events more then once (or you may need to replay them, failures etc). Thinking
Publishing events in your architecture? What type of event are you publishing?
Knowing the different types of events you can publish and tradeoffs with each one can help you remain decoupled, and avoid a ball of mess.
Visual with resources 👇
When building event-driven architectures, your architecture will consist of many messaging/event patterns.
Point-to-point, pub/sub, streaming, integration patterns and more...
Here is a visual to dive deeper 👇
Building event-driven architectures? Understanding event streaming can help.
What are event streams? When are people using them? How do log based brokers work? Get value from data in real-time.
My bite sized thoughts 👇
Local cache copy vs requesting data ⭐️
When consuming events in event-driven architectures, sometimes the consumer may need more information.
In this new bite sized visual we look at two patterns. Requesting for information or keeping a local copy, and the tradeoffs between
Mixing techniques from domain driven design with event-driven architecture, can help us architect cleaner solutions.
Find boundaries, communicate with messages, don't expose implementation details, context mapping, can help.
My visual to dive deeper 👇
Something I wasn't aware of when I starting to build event driven architectures, the difference between commands and events...
Understanding is extremely helpful, commands represent the intent, events the facts...
Dive deeper with this visual 👇
Tomorrow I will be launching
@event_catalog
v2 ♥️
And I thought I would share a few thoughts...
This project combines my passion for event-driven architecture, domain-driven design and developer tools.
EventCatalog will allow you do document and version domains, services and
Event driven architectures, domain-driven design and Conway's law have many things in common.
Our architecture is reflected by the teams that build it, so how can EDA impact this? Does org structure impact EDA delivery?
Visual, notes and resources 👇
"Change data capture" is a super powerful way to listen and observe changes in your database.
Couple this with event-driven architectures and we start to see some powerful patterns ⚡️
Bite sizes visual and resources 👇
Event driven architectures and visualisations this is the kinda thing I think about...
Ways that can show what is happening, and ability to easily test / explore them.
This is my EventBridge Bus in AWS, rules and targets.
I believe visualising EDA can enable so much 🤔🤩🚀
I just released "awesome-eventbridge"
An open-source list to help learn and use Amazon EventBridge ⭐️
📗 Resources
📹 Videos
🛠 Tools
Feel free to contribute! 🌎🚀
Inspired by
@alexbdebrie
"awesome-dynamodb" GitHub repo ⭐️🙏
#serverless
#eventbridge
Had an idea last week to develop a website to help developers find React resource. There is plenty of GitHub repos that share info, but I wanted something visual...
The project is moving in the right direction, a few more things to do...
#React
#JavaScript
Been using ChatGPT to write bed time stories for my kids, moved it to an event driven application 🥹
Every night my schedule triggered, generates story and emails me a link.
Schedule > Story (in DDB) > Event raised > SNS (Email me link).
For parents, will open source soon ❤️
Just been reading how McDonald’s are doing event-driven architectures 🍔😍
⚠️ Importance of standards
📄 Schema registry + validation
🌎 Event Gateway Pattern
🧑💻 Custom SDKs
✅ Defined flow of events
Here are some thoughts (with blog links) 🧵👇
I have been thinking about this for awhile...
I think event-driven architectures are fantastic, coupled with DDD it's a match made in heaven❤️
Implementation is a journey, EDA doesn't happen over night, it takes time, care, and thought...
My thoughts 👇
When scaling event-driven architectures, common issues arise
Thought it was me... but speaking to folks these issues keep coming up over and over...
e.g Discoverability? Leaky events? Coupling? Implementation > Behaviour?
Visual & resources to help 👇
Been using AWS Eventbridge for just over a year now, and I have to say Event driven architecture is the future for sure.
The ability to extend your architecture, add business rules and keep a decoupled architecture is awesome.
Eventbridge is the icing on top!
Publish messages, process them at desired rates, process them in parallel, pull messages from queues...
Messages can help you scale, remain decoupled and introduce interesting patterns ❤️
Visual & resources 👇
EDA Visuals to PDF it coming along ❤️
Documenting containing my notes and visuals about event-driven architectures, a few more things to add/change then I will share with you all 🚀 (for free ofc)
I just launched CloudCatalog!
An open source tool to help you document your AWS architectures.
🔎 Discover resources
📄 Document resources
👥 Assign owners
⚡️ Powered by markdown
🤩 And more...
If you find it useful, a retweet would be awesome! ♥️🌎
Private vs public events in event-driven architecture...
Depending on your boundaries, can depend on what events you raise. Internal or integration events, both have very different characteristics and maintenance requirements 🤔
Visual to dive deeper 👇
If you are new to Serverless or message brokers, you may be wondering why use them? 🤔
They can help with decoupling, reducing back pressure, processing messages in parallel and much more....
If you want to know more, I have a visual on it 👇
Today I got accepted into the AWS Community Builders program for serverless ⭐️🚀
I'm looking forward to continuing to share my thoughts, ideas, and experiences with the community 🌎
#AWSCommunity
#AWSCommunityBuilders
Event design is important and can be important to know the difference between event types 🤔
Notification, ESCT, Delta, Naming conventions, service events/custom events, and more...
New Serverless visual for you all 👇
ServerlessLand continues to grow, we now have over 450 patterns, 70 step functions workflows, 50 snippets, videos and much more.
I'm working on a new feature I'm hoping to drop this week that will let you search this all ❤️
Hopefully this will help 🙏🚀
How you can use events to migrate between legacy applications to your new architecture.
Careful about conforming to legacy domain models, context mapping can help. Domains may evolve maybe time to validate them?
Visual and resources 👇
I'm super passionate about event payloads when building event architectures.
But what happens when we expose domain information? Why care about ubiquitous language? What are domain and integration events?
My thoughts and extra resources to help 👇
Diving into event-driven architectures? You will see the terms choreography and orchestration.
Knowing the difference can help, and as you dive into domain driven design you might find yourself in situations where the patterns emerge
Visual to help 👇
"AWS re:Invent 2021 - Best practices of advanced serverless developers"
I love this talk by
@julian_wood
, if you are into Serverless then I highly recommend it.
Some great info about Event Architectures too, here are some of my takeaways from the talk 👇🧵
The one thing I love about event-driven architectures is the agility they can provide us.
Isolation of failures, publishing future events, team topologies, growth, scaling... can EDA really help all of this!?
I believe it can.
Visual and resources 👇
What are internal vs external events in event-driven architectures?
I like to group services and events by domains. Depending on where my events live, depends on what goes inside my events, how they are managed, and the cognitive load required when dealing with them.
What are
ServerlessLand is an open source project to help you learn Serverless and EDA.
As 2022 ends, we have over 420 patterns!
⭐️ 74 TypeScript
⭐️ 95 Python
⭐️ 98 Node.js
⭐️ 32 Java
⭐️ 16 .NET
⭐️ 18 Rust
Covering CDK, SAM, Serverless Framework and Terraform.
What is an Event bus? How do they work with Amazon EventBridge? 🤔
What is the default bus? Why create custom buses? Understand bus topologies, cross account events, AWS integrations, and more...
Here is my bite sized visual with resources to help
Been thinking about explicit vs implicit events when building event architectures....
Here are some of my thoughts on the importance of making your events explicit and side effects that can happen if not.... 👇
Here it is, my first serverless visual (of many)... first focus is on EventBridge 🚀
Let's start with some basics, EventBridge Targets. What are they? How they help with fan out in our event-architectures, and much more...
Visual with extra resources 👇
Was fun building this website using
@Netlify
,
@gatsbyjs
and
@github
.
Thanks to
@gatsbyjs
& it's plugin system I managed to get something up and running extremely fast 🙏👍 and performance out the box 🎉.
Avoiding the "big ball of mud" in event-driven architectures
When building event-driven architectures it can be easy to fall into a "big ball of mud" architecture with lack of design and considerations ⚠️
Architectures that are a "big ball of mud" lack structure, models are
What makes a successful event-driven architecture?
For me it's about developing an understanding, standards, patterns and focus on governance 💯
Combined, these can help you build successful event-driven solutions 🚀
🆕 Visual and resources 👇
Had a couple of hours to kill today, trying out the AI Vision API, and generated a AI serverless pattern generator 🤩
✍🏻 1. Draw your AWS serverless pattern
🤔 2. Pick your IaC option
🤖 3. Hit generate!
Using your drawn architecture, the code is generated ❤️
Demo 👇
🆕 11 new serverless snippets added for API Gateway
If you use API Gateway then checkout these CloudWatch queries, all open source and shared by community ♥️
Check it out 👉
My EDA visuals "book" now has over 40 visuals to help you all learn about event-driven architecture.
I started visuals in Jan, and it's awesome to see them helping folks learn more about EDA 🙏❤️
All free to help spread the word about EDA, enjoy 👇
What is publish/subscribe (pub/sub) messaging when building event-driven applications?
Simple and powerful. One of my favourites, for many reasons.
Visual and resources 👇
Introducing ServerlessLand Search (v1)
All resources are now indexed, allowing you to search for serverless content faster ⚡️
🎨 400+ Patterns
📹 400+ Videos
📹 70+ Workflows
🧑💻 50+ Snippets
Hope is helps 🚀
Get started 👇🧵
New visual out, this one focused on event-driven architecture and teams cognitive load ⭐️
It’s inevitable that software evolves over time, requirements change, domains within businesses change.
At the start of a new project, it seems relatively simple for us to get a holistic
Diving into DDD, Messaging and Event Payload standards recently, came across some great work by
@mathiasverraes
, some interesting patterns and thoughts 👇
How will your message reach your consumer?
Here are some short notes to understand the difference between at most once, at least once and exactly once delivery.
Visual and resource to help you all 👇
Currently writing more visuals around AWS Services.
Hopefully this format will give you all another way to learn, with bite sized visuals + AWS Services.
First service I will focus on is EventBridge (ofc...), I will be writing these types of visuals to help you all 👇🚀
I honestly think it's incredible what people are building with Step Functions 😍
📦 CI/CD
📖 Machine Learning
🔎 Auditing
📃 Data processing
🤝 Human approval flows
📦 Microservices
✅ Integration/e2e testing
🌎 and much more...
Love to hear what you are using them for?
Into Amazon EventBridge?
I'm writing an open-source tool to help you quickly write input transforms 🤩
Added some new features:
✅ See example output from your templates
✅ On the fly validation
✅ Console Widgets
✅ And more...
Demo 👇♥️
Using Event-Driven Architectures? 🤔
Want a quick way to document events, services and visualise them?
Here is a sneak peek of the new "Visualiser" feature coming for 🚀
Did you know there are tools out there to help you document your event-driven architectures?
Something I've been super passionate about the past year. Discoverability as you scale can be super useful.
Here are notes and links to help you 👇
Raising events within your domain is different to when you raise events that integrate with domains.
You may be using the same broker, events may look the same, but the relationships are different.
Visual to dive deeper 👇
I just launched EventBridge Canon!
A new AWS EventBridge Tool ❤️
✨ Save & Publish Events
✨ Replay Previous Events
✨ Import/Export Events
✨ Data stored locally
✨ Event Editor
If you find it useful, a retweet would be awesome! ❤️🌎
#aws
#serverless
Want to learn more about event-driven architectures on AWS? This new resource is epic! 😍
📦 EDA Patterns
🔃 Client <-> Backend events
🗂 Data processing
❌ Handling failures
🎁 And much more
Best of all? It's all free!
Awesome first day at
@awscloud
, the Serverless DA team are a great bunch!
Can't wait to see what I can learn and how I can contribute to help you all learn too! Going to be epic 🚀⭐