I have a controversial blog post to write today, but I’ve written controversial ones before. Ready? I want to talk about Elon Musk.

Elon fascinates me. There are people on both sides who claim he’s a huge genius, or a blithering idiot. I’m not sure which one he is, as I’ve seen evidence for both. But what’s truly fascinating is the same data point is sometimes simultaneously evidence for both.

Confusing? Let me explain.

So there was a story, probably apocryphal, that after he bought Twitter he went into one of their datacenters. Someone pointed to a rack and said something like, “That’s our hot spare cluster failover.” So Elon walked over and pulled the plug so the whole rack went down. Nothing happened to Twitter, of course. The site kept working, and he proclaimed, “See? Useless. Decom that, we just saved $X.”

Now of course, the site reliability and systems engineers all went apoplectic and sputtered about high availability and SLAs and MTTR and so forth. I did too, at the time. And we were not wrong.

But…

Does Twitter really need high availability?

Read that again, slowly and carefully.

Does Twitter really need high availability?

“It’s a global shared highly-used platform!”

Sure. But it’s not a nuclear power plant, an air traffic control system, or a wastewater control station. What really happens if Twitter goes down for a while? Lost ad revenue, minor business disruption, reputational risk? Look at the case studies and you’ll find the impacts are nowhere near as bad as everyone seems to think. In October 2021, Facebook, Instagram, and WhatsApp all went dark worldwide for about six hours. It was a big deal for a day, but by the next day everyone went back to doom scrolling. There have been major outages on other global nonessential systems too, like Sony PlayStation and even the cloud hyperscalers, and they’re all still here and doing just fine.

Look, I’m not saying there wasn’t a repercussion to that decision (if it actually even happened that way). Twitter, now X, did suffer several high-profile outages after the acquisition and the layoffs that followed. Users noticed, the press noticed, and the jokes wrote themselves. But X is still here too. Whatever problems the company has had since, downtime isn’t the one anyone points to.

So was Elon just completely clueless on the IT fundamentals of high availability? Or was he making a business calculation that the cost of running a hot standby instead of a warm or cold one wasn’t worth whatever extra nines Twitter would get on its uptime?

Honestly? I have no idea.

But I will say this. I’ve had a gradual realization that there was potentially a significant positive business financial impact to what seemed like a terrible engineering decision at the time. It has truly impacted my way of problem analysis.

In some things, this matters. I’ve never been in a car crash where my airbag deployed, but I will happily pay whatever extra the car company charges me to make sure it’s there and works. The risk impact is too great. But on the flip side, have you noticed many new cars don’t come with spare tires any more? Because most of the time you just use your cellphone to call roadside assistance, so why bother paying extra for that tire and jack?

Learning to bring business and financial analysis into your engineering decisions is, in the modern world, one of the keys to whether your organization or company can be successful. Engineers love to ask how many nines we can get. The better question is how many nines we’re actually paying for, and whether they’re worth it.