I have no information about how this incident came to be but I can confidently predict that people will blame it on greedy execs and sloppy devs, regardless of what the actual details are. And they will therefore learn nothing from the details.

New blog post:

@ShriramKMurthi In fairness, 2016 was three lifetimes ago.

RT @thiagoghisi: Alright, here it is, my last "2024 Favorite List" of the year: Alright, here it is, my last "2024 Favorite List" of the ye…

(1) How can we prepare to better respond to future events *that we didn’t anticipate before but now know about*? (2) How can we prepare to better respond to future *novel unanticipated events*? People always ask about (1). The hard part is getting them to think about (2).

AI has yielded enormous gains in the process of transferring money from venture capitalists to NVIDIA.

Questions you can always ask in an incident review: What made this particular incident hard to deal with? What skills helped the responders diagnose and remediate?

RT @kdrum: Why don’t we do more prescribed burning? An explainer.

New blog post:

1. Our models of the world are always wrong in important and dangerous ways. 2. The automated systems we build embed these models within them. Therefore, 3. our automated systems are forever at risk at getting the world wrong in dangerous ways.

Annotated timelines are underrated

No matter how well-defined your processes are, they won't help you resolve ambiguity.

An ounce of adaptive capacity is worth a pound of prevention.

@Geekonomicon @jarvis_best @danielgothits Toronto and Vancouver are ridiculously expensive.

We jump from “what” to “why”, but it’s the “how’ that we really need to understand.

@martin_casado I don't know what else to say at this point:

@martin_casado In my experience, software engineers generally don't know how much load an individual pod/instance can handle, and they certainly don't adjust scaling policies whenever they see a performance regression! Closed-loop autoscalers mean they don't even have to think about it.

@martin_casado I’m confused by that response. Autoscaling is closed-loop: it uses feedback to adjust cluster size based on deviation from a target value.

RT @ShriramKMurthi: That's most uses across ~1.5mo. In most cases I'd gone to search first. Is any of this life-changing? No, but it's made…