@notshenetworks
They had trouble paying house cleaners and nannies? What a joke. Cleaning the house was a group effort in my family and daycare was sitting on the front doorstep after school waiting for my Mom to get back from work. My parents sacrificed their own needs for our groceries.
Jeff Bezos once said if you mention the words "work/life balance" during an interview at Amazon you are a no hire.
After a month at
@MongoDB
it is 5PM on Friday and I do not feel like there are a dozen things I have to get done before the weekend starts.
I can get used to this.
@rickyschroder13
A mentally disturbed kid shoots three people, killing two of them and you fund his bail to get profile views. The country we live in allows it, but the fact that you do it is disgusting.
I love contributing to projects that impact the world. Last year I helped the Government of India standup the world's largest COVID vaccine registry and it has maintained 100% availability since launch, peaking at over 150K req/sec. I am so proud that my work is saving lives.
My 2023 advice for all young engineers:
Respect a running system. There are reasons it works that way, most of which will be uncovered when you try to replace it.
Make incremental changes, don't burn the building down.
No one way doors. Never be unable to rollback.
@granthyarbrough
@notshenetworks
Agree, but driving an Audi and hiring nannies and housekeepers are choices that have little to do with affordable daycare. Struggling to pay those bills is a choice.
Serverless means:
1. Never having to patch another OS.
2. Never being paged when a rack goes down.
3. Never needing to rollout software on a hundred nodes.
4. Never paying for idle infrastructure.
5. Never losing business because you can't scale.
Last day at AWS today. I am very proud of the amazing community that we created for
@DynamoDB
over the last 7 years. I will be taking a short break before resuming my NoSQL journey right back where it began as Director of Developer Relations for Strategic Accounts at ⚡
@MongoDB
⚡
When discussing modernization, remember the "legacy" service is the one that is actually working.
Always respect a running system.
Make incremental changes, don't boil the ocean.
Measure the impact, understand what happened.
No one way doors, never be unable to rollback.
Most seem to miss this young woman's point. She expects a degree she was groomed to obtain by our "for profit" education system should qualify for a job with better pay than sushi server. Instead she is saddled with debt and cannot take the entry level gig. She has a legit gripe.
Costco’s new CEO started at the company operating forklifts, and worked his way up to CEO after 40 years
Meanwhile this 25 year old gen Z woman expects to be making $150,000 a year because she has a marketing degree
#NoSQL
databases include both attribute names and values in every item they store. In one design review today we estimated 70% of
@DynamoDB
IOPS and storage is being consumed by attribute names. Short names will eliminate 14K WCU at peak and reduce table size by 3.5TB.
I hear a lot of criticism of the work I did on DynamoDB Single Table Design these days. I even saw
@alexbdebrie
apologizing for being an advocate the other day. The core concept behind STD, however, has always been the same as it is for all
#NoSQL
databases.
What is accessed
Life no longer shines for my family as it did only 2 days ago. I am not sure it ever will again. My youngest son has lost his battle with mental illness. It appears his passing was unintentional, but we will never see him again. My heart is broken. We love you forever Henri.
My entire career I have been told I am wrong.
10 patents in virtualization, complex event processing, and microprocessor design,
2 products acquired by Fortune 500 companies.
WW Technical Leader - NoSQL Services
@awscloud
Your critics are wrong, listen to your own voice.
@Jules31415
The problem with this statement is you are assuming 96% of Americans agree with you. The reality is its just the intolerant ones that do. All of you are welcome here just like everyone else, but you need to understand most of us actually do not agree with you.
I almost missed my career in tech. I dropped out of school in 1983 after a stupid BB gun battle blinded me in my right eye and benched the idea of using ROTC or football to pay for it. My uncle got me a job in the parts warehouse at NCR in Mountain View.
At NCR I designed my
"This session was the best ever, by far. If there is a Voyager 3, a video of this session should be sent in the ship to tell other species how advanced we are."
Best feedback comment ever. Video should be up soon.
Almost 30 years later the best bang for the buck programming language is still Java. Managed framework, robust API, easy to learn, efficient and performant even if it is a memory hog. Rust scales a bit better but the mental tax is high. For most workloads Java is a solid choice.
There is a story about how when Amazon realized that 70% of their access patterns were key-value queries for a single row of data they decided to move to
@DynamoDB
. This story is true, but making the other 30% work was my team's job.
#SingleTableDesign
Talked to a >$1B retailer based this morning that is running all enterprise operations on
@DynamoDB
for $1,200/month. Provisioned for peak load with legacy technology they would be paying more than 10 times that.
Pay for what you use and no more with
#CloudNative
#NoSQL
In November 2018 I delivered a session at re:Invent on the NoSQL design patterns my team developed at Amazon. I had one follower when I started tweeting about that content. Today I have almost 11K and I am blown away by the support that I have received over the years.
Thank you.
Next week during
@DynamoDB
Office Hours I will model the Wordpress default database schema and take questions live.
Tuesday 6/30
@11
:00AM CDT (GMT -6).
I never graduated from college and my first real job was construction laborer. If I listened to folks who said I can't do it, I am not good enough, or the risk of failure is too high I would still be digging ditches. Glad I chose not to and screw y'all who did that. 😛
This life lesson was a challenge for me early in my career, but it is the most powerful one I have learned:
Success happens to those who are willing to embrace an opportunity even when you are not sure you can do it. Do not fear failure, learn from it.
I was recently asked why AWS AppSync does not support
#SingleTableDesign
without a custom resolver. When AppSync was built there was a Senior PE who thought those patterns made no sense so they designed the DynamoDB integration without me. The result contradicts best practices.
Earn Trust. That was always my favorite LP at Amazon.
Always work in good faith with your customers and team. Embrace truth and don't filter reality to match preconceived belief. Be open minded to new ideas and let everyone contribute.
Trust stops when minds close.
So it finally happened. I had a Design Review with a customer the other day who is leaving DynamoDB because their cost has gone through the roof as they scaled. They listened to one of the most vocal critics of Single Table Design and went with a multi-table pattern instead. They
Cleaned out the closet and found this award I received for leading the Amazon NoSQL Blackbelt team. We educated 25K developers, migrated thousands of services to
#NoSQL
, and shutdown 3K Oracle server instances. Before that
#SingleTableDesign
did not exist, now it is everywhere.
Talked with a customer that recently turned $5K/month spend on EC2 instances for a legacy
#NoSQL
cluster into $1K/month for
@DynamoDB
usage and repurposed 2 people from maintaining the cluster to working on the product. Net savings of $500K annual.
TCO matters. Choose wisely.
Been digging into
@AWSAmplify
, GraphQL and AppSync lately. Here is an example of how to build a complex object behind an AppSync API with a single
@DynamoDB
query using a Response Mapper. Pretty cool stuff.
#NoSQL
#SingleTableDesign
Come join me at re:Invent and learn how to model complex relational data in NoSQL. I will be discussing design patterns and best practices for DynamoDB, modeling for Document versus Wide Column databases, as well as diving deep into complex single table design.
Just found out a patent I wrote on index key overloading was granted a few months ago. This is number 10 for me, and my first specific to
#NoSQL
technology.
I would never have imagined when I wrote my first line of code I would still be inventing at 57.
I love it when non-technical people tell me that the content I submit for a technical session will not be interesting for the audience. In 2018 I was also told this before I delivered the most popular breakout session in re:Invent history.
If you want to guarantee mediocrity in your workforce then simply treat all your employees equally and your best and brightest will leave and go someplace where they are recognized.
@BradHensonPro
Not every foul will be seen every time. The fact is an official threw the flag on an obvious foul. Complaining about that is ridiculous.
You cannot promote everyone, so if all must have a "growth plan" then the HR plan is move up or move out. Growth happens when people enjoy work and return value to the business. Don't knock IC's who are good at what they do and don't want more, you cannot scale without them.
I had the amazing privilege last summer of designing the schema for one of the world's largest COVID-19 vaccine registration services which is now in production hitting 7K TPS and handling over 1.2M registrations and 1.5M vaccinations daily.
@DynamoDB
#NoSQL
#SingleTableDesign
1 hour until I present live to a real audience for the first time in awhile. I have gotten used to talking to my webcam...feels like the first time all over again.
People who argue against
#serverless
talk about vendor lock in and cost of AWS Lambda. People who actually use it glow about how much they are saving and how agile their teams are without having to maintain physical infrastructure.
I trust user testimonials over FUD.
AWS Lambda switched to "per ms billing" on Dec 1st 2020.
Despite continued adoption and shifting more workload to Lambda, our December bill for our Lambda workload will come in at ~40% or less compared to previous months.
Keep it coming,
@awscloud
.
#serverless
for the win!
Things I get now that I work with
@MongoDB
:
Indexing JSON values and list attributes!
Full Text and Geospatial indexes!
Aggregation queries!
Compound indexing!
Things I lose:
Manual write sharding
Weird DynamoDB JSON
Built for no purpose databases
I can live with that.
Buy this book now! This is the first useful guide on NoSQL data modeling. Everything else I have seen is full of bad assumptions and anti-patterns. Alex did a fantastic job documenting and enhancing the common NoSQL design patterns we use at AWS.
A common critique of NoSQL is that optimizing the data model for the access patterns of the application is hard to do unless they are well understood.
If you are coding without understanding what queries you need then you need to be writing user stories, not code.
For many people "working hard" means spending a lot of time ensuring management thinks they are rock stars. The ones who actually are spend more time helping peers and customers than messaging up the org. If you want to know who your rock stars really are then ask your team.
An NFT is nothing more than a web address that points to a server that hosts some content. There is no guarantee whatsoever that content will always be available or that it will not change. Buying an NFT is paying for a URL string. Dumbest idea ever.
With well over 200K views the 2018 DAT401 breakout is by far the most popular session in
@AWSreInvent
history. Thanks to everyone who has taken time to watch and learn about modeling data for
@DynamoDB
.
#SingleTableDesign
#NoSQL
I joined Twitter in 2012 when I was at a trendy startup and all the execs had to not be lame. In 2018 I had 1 follower when I presented on
#SingleTableDesign
at re:Invent. Now I have almost 10K.
I am blown away that you all think my words are worth your time.
Thank you.
I am tired of technical evangelists explaining things they do not understand by delivering canned content and running through lists of talking points then stumbling on questions.
There is no substitute for knowing what you are talking about. Learn before trying to teach others.
Finally got around to setting up a Pi-hole. Web browsing was noticeably faster immediately and 24 hours later thousands of requests blocked on my home network. Highly recommend doing this.
@DynamoDB
delivered an impressive 89.5MM peak TPS for Amazon on Prime Day. The fact that it did this while also delivering untold IOPS to the rest of the world is mind blowing. There is no other
#NoSQL
technology in the world that operates at DynamoDB scale.
An account team asked me to deliver a 10 minute brief contrasting
#NoSQL
and RDBMS for their customer.
Best feedback quote ever:
"Felt like I was back in school watching a mind blowing lecture that literally shredded everything I thought I knew about NoSQL and
@DynamoDB
."
Ran into a young SDE the other day in the vaccine line who told me all about
#NoSQL
was and why it was better than RDBMS. He talked so much I was not able to get a word in sideways so I just listened and smiled.
Airline reservations and flight operations services leverage complex relational access patterns. Today on
@DynamoDB
Office Hours I will demonstrate how to model an airline reservation system based on the following ERD.
today
@9
:00AM PST
@JesseNews3LV
Chandler Jones, Jimmy G, and drafting an injured DE with his only first round pick pretty much sums up Ziegler's career as a GM.
He was awful.
3 hours until I go on stage to talk about how
#NoSQL
will eventually rule our data universe, the virtues of single table design and why Cloud Native NoSQL is the only choice for modern apps.
@dynamodb
#reInvent2019
#AWS
Here is my latest contribution to the re:Invent library. As always it was both an honor and a pleasure to speak at this amazing event. My session was oversubscribed once again. Thank you to everyone who takes the time to listen to my message.
Join me this Wednesday for another
#NoSQL
data modeling session on
@DynamoDB
Office Hours. This time I will be using the web based data modeler to design an Online Banking service.
#SingleTableDesign
Wednesday 5/5
@9
:00AM PDT
It is easy to throw out the old stack and start building something better, but the devil is in the details. The best migration plans avoid problems by making incremental changes over time to move to a target state.
Always respect a running system.
Next week on Office Hours I will be modeling an Online Banking service live and taking questions on
@DynamoDB
.
Send me an access pattern you would like to see for this schema and I will model it.
Friday, July 24
@12
:30PM CDT
Grief is not constant, it is intermittent and polymorphic. A moment spent quietly contemplating what could have been. An explosion in your brain overriding every input while you scream at the world from inside your own head. Losing a piece of your soul is hard.
Join
@rafalwilinski
and I tomorrow on
@DynamoDB
Office Hours. We will be modeling a simple customer order service using the following schema and exploring the new
@dynobase
table modeler.
Wednesday 11/2
@9
:00AM PST
Reviewed a timecard reporting service yesterday that will be supporting ~2K field engineers. The team is currently billed $3K/month by their outsourced IT provider for a redundant RDBMS solution. The workload is going to be free tier on
@DynamoDB
.
25WCU/25RCU/25GB
Always free.
Tomorrow on
@DynamoDB
Office Hours
@alexbdebrie
will be joining for a discussion on Single vs. Multi-table Design, when Elasticsearch makes sense, managing concurrency, pagination and more.
3/24 - - 9:00AM PDT
AWS PM's tell me I am wrong about how important UI/UX is to them, but nothing speaks louder than the volumes on uncurated content that is spread around the docs. You never really know if what you are reading is what you will find when you open the UI.
I often hear how hard it is to model data in
#NoSQL
compared to RDBMS, but nobody ever talks about optimizing SQL. Usually when I ask a group of SDE's how to use a query analyzer I get blank stares.
In NoSQL all you need is a spreadsheet to model data. It does not get easier.
Non-relational data is a term invented by marketing people to explain to other marketing people how
#NoSQL
is different from RDBMS.
All data is relational. Developers know this. What developers need to know is how to model those relationships efficiently in NoSQL.
@MongoDB
Distributed SQL is like Henri Ford deciding to give customers what they wanted and breeding a faster horse. Since 2014 per core CPU performance is flattening, but storage costs keep falling. Increasing the time complexity of queries is not the answer.
#NoSQL
@DynamoDB
In 2017 we created a Distributed Lock Pattern for Amazon retail that was released as the Amazon DynamoDB Distributed Lock Client. Recently a customer asked if there was a similar library for
@MongoDB
, so I forked the project and abstracted the data layer.
@juancpgo
@MongoDB
That is the foundation of Amazon HR philosophy in a nutshell.
Amazon is full of amazing people who care about their peers and employees, but the machine is designed to maximize output and treat employees as commodities that are constantly reassessed and revalued.
My personal Leadership Principles are derived from
@patricklencioni
's Five Dysfunctions of a Team. I read this book two decades ago and it still resonates today:
1. Earn Trust - Communicate openly and honestly at all times. Always remember trust is earned, not given. Don't play
GraphQL apps are usually really good at burning infrastructure to access data inefficiently. They don't have to be, but doing it right requires the same knowledge as using any database properly. Efficiency has no shortcuts.
The best technical evangelists are doers, not just talkers. If you want to succeed in devrel then you better be able to walk the walk when its time to code. Make time to solve real customer problems every now and then. Keep your skills sharp.
ACID is an anti-pattern within a distributed system.
1. Requires process synchronization.
2. Potential deadlocks/brownouts.
3. High overhead.
Systems that scale are non-blocking. Use optimistic concurrency, versioning or event sourcing patterns to manage consistency instead.
Using NoSQL incorrectly is becoming so normal that anti-patterns are turning into tribal knowledge and the flood of bad guidance that permeates the Internet is preventing people from realizing the benefit.
If you want good results then learn how to use a tool before you use it.
I have never said the RDBMS cannot scale. I have always said that joins are more expensive than index lookups so try to avoid them.
It's not my opinion, its math. Indexing documents in a common collection is faster than joining tables. Even Edgar Codd knew this.
#NoSQL
@MongoDB
Do you leave your car running when it's parked in a garage? Why are you paying for your IT infrastructure when your customers are sleeping? Serverless means pay for only what you use and save up to 90% on IT cost.
@DynamoDB
@AWSLambda
@awsserverless
This came out of the blue from one of our Senior Developers here at
@MongoDB
and it really made my day. Feedback like this is amazing validation. I am so honored that really smart people like this find value in my work.
Access patterns are crucial part of single-table modelling, they should be the lens through which you're looking at your data.
Upcoming version of
@dynobase
will make that clear.
Tomorrow on
@DynamoDB
Office Hours I will be modeling a Hospital ER Diagram with associated workflows. As always I will start with a set of contrived access patterns but feel free to suggest your own.
Wednesday 11/25
@9
:00AM PST
Here are the access patterns and
#NoSQL
schema for the Amazon Kindle Collection Rights Service. Running 9 distinct queries millions of times a day on RDBMS was a massive waste of CPU. Switching to
@DynamoDB
saved big $$.
Need to support low frequency ad hoc queries against operational data? Deploy with
@DynamoDB
->Firehose->S3<-Athena for the best of both worlds. Fast and consistent performance for your OLTP workloads with support for ad hoc OLAP on the fly.
It does not get better than this.
@MissAmyTobey
I was never a great coder. I failed every time I was put through one of those ordeals. My best code was refactored by someone else before it went to prod.
I have 9 patents, wrote the first VM hypervisor, and invented Single Table Design for DynamoDB.
I am the idea guy.
#SingleTableDesign
has two basic concepts, Polymorphic Tables and Access Pattern Overloading. The first is about storing all data on one table in related groups of entity objects. The second is about indexing objects on multiple dimensions to group and sort them in many ways.