EVIL ARTIFICIAL INTELLIGENCE HYPERGOD VERSUS HUMANITY: CHOOSE YOUR FIGHTER
@shinydan It's not just some herbert with a shovel, it's the maps that got misfiled with the city, it's the telco that changes cable routing without upstream notification, it's it's it's it's it's....
@shinydan
Years ago, a friend of mine got to watch the triply-redundant, geographically disparate fiber runs he'd meticulously planned to guarantee continuous connectivity to an industrial plant all vanish in the span of a second, when their provider - despite those fiber links leaving the plant in 3 different cardinal directions and going kilometers out - decided to bring them back together, 10km past the edge of his map, under one overpass to save money. Which is where the backhoe found them.
@mhoye @shinydan That’s happened to me, too, and with four WAN circuits from different providers! We paid a *lot* of money for physically diverse paths. Turns out there was a stretch where they all went through the same fiber bundle.
Then a second time at the same site, we got notifications from all of our carriers about router maintenance happening at exactly the same time. Turns out three of them leased capacity from the fourth, and all four lines terminated on one router. Super great.
@mhoye I was at a pub one Friday afternoon with a bunch of people including a Google Ops guy. He got a message in his phone and went “That’s weird, every undersea cable landing on the east coast of Japan just all went dark at the same time.”
That may well have been the earliest sign of the Fukushima disaster.
@mhoye I used to work in an electric company call centre. No matter how much money you put into the National Grid, there is always some herbert with a shovel who can accidentally cut you off.