Post
My first job was building out the first mega-datacenters. 2005-2007, I was a datacenter assistant monkey working from Google working somewhere in the Chicago suburbs, swapping out hard drives and ram and writing shell scripts, as myself and my friends unknowingly laid down the prototype for the kinds of datacenters we all see today.
And so it is with some significant expertise that I say:
Fuck datacenters. Datacenters are an anti-pattern.
What if: instead of solving the "last mile problem" we consolidated power into a few hegemonic rich get richer interests, broke the end-to-end nature of the Internet with NAT and asymmetric bandwidth, offered "broadband" via satellites at hundreds of times the cost and proposed "datacenters in SPACE" to maximize latency even more!? All while shitting on peer to peer protocols and mesh routing innovations for the past decades, because that might marginalize our profits for our precious "shareholders"?
Such a game plan, truly, dumb founding! Much "intelligence", very artificial.
(sarcastically laughs into oblivion like a super villain)
@cwebber Indeed! I have several issues with datacenters.
First, using large amounts of water for cooling is NOT required. Evaporative cooling is NOT required. The fact that every so-called journalist writes about datacenter projects without calling out the megacorps who simply don’t want to pay for non-evaporative cooling just makes things worse.
I have servers colocated in three different datacenters in three different US states. None use evaporative cooling. For years I had a quarter rack in a datacenter in Phoenix, Arizona, also without evaporative cooling. In Phoenix!
Sure, people who like to make excuses for megacorps will say that high density compute like what AI companies want to build can’t be cooled without evaporation, but that’s simply not true. It can be done, but it costs much more, and AI companies don’t care about 10+ year ROI.
The articles that don’t point out that all of these projects are bribing - er, negotiating cheaper power and water from municipal sources that are driving residential prices higher are doing everyone a disservice.
All of this is one of many failings of capitalism. When a single corporate entity can demand so much of a service’s output that it can adversely shape the costs of basic, required services, there’s something wrong. The fact that there’s no governmental protections from this kind of abuse is highly problematic.
It sure would be nice if journalists pointed out that the results of megadatacenters aren’t inevitable, but chosen based on how much hurt can be pushed on to everyone else.
@AnachronistJohn @cwebber the data center water use scandal is just fake outrage. There are other industries that are using orders of magnitude more water
@slyecho @cwebber Do you have some examples and numbers to share?
Also, it’s not “outrage” to not want your local municipal water prices to go up, for your water quality to go down, for your aquifiers to run dry and/or for your area to run out of water, or anyone else’s. Saying that someone else is doing something worse so that doing something bad isn’t as bad isn’t really a persuasive argument.
@AnachronistJohn @cwebber here is one article that has a lot of examples and references: https://blog.andymasley.com/p/the-ai-water-issue-is-fake
@cwebber And datacenters are overbuilt. If we didn’t literally cripple our residential Internet, we could run datacenters from home.
I had a colo go out of business while in possession of my servers, so I ran a full time email server for a few hundred users out of a house. Between the UPS and the auto-start generator, power’s about as good as a datacenter’s. The weak point was that I had to route public, static IPs through a tunnel to another colocated machine. If residential and non-residential Internet didn’t have a literal order of magnitude or more price difference, I wouldn’t even need the other colocated machine.
The point is that some of us are using the leftover crumbs of what datacenters are for because we have few other practical options. It sucks, and it’s sad. Now it seems likely that even those crumbs will be harder to find. It’s not a good direction.
there is this character/mask of turin, part of the commedy of art, name is gianduia (which is also as gianduiotto a typical candy) ...
he is famous for hiding money in somebody elses pockets ...
i think of it every time i hear the word cloud ...
🤣
@cwebber Data centers will be around till someone writes software making AI computation in a distributed fashion.
there is this caracter/mask of turin, part of the commedy of art, name is gianduia (which is also as gianduiotto a typical candy) ...
he is famous for hiding money in somebody elses pockets ...
i think of it every time i hear the word cloud ...
🤣
@cwebber datacenter bad, colocation good
@cwebber I did this in Oregon. The town looks nothing like its old self.
@cwebber
huh
interesting.
about that same time;
had 3 interviews with the goog
as a 'linux beige box wrangler'
at a big campus dc in northern va
my linux clue-kit was deep & shallow at the same time
they passed me over
I was bummed, bigly
then it all changed
@cwebber The main motivations for using data centers are security, reliability, flexibility, and efficiency. A data center can manage power consumption much better than your typical on premises installation. Infrastructure can be shared by many users, which allows dynamic scaling without having to over provision on premises. It also allows users to share expert reliability and cybersecurity engineers.
@cwebber I agree with your premise. However, realistically, convincing businesses to return to on-prem is challenging. It's not just about management – maintaining, securing, and procuring on-prem hardware is a significant expense and effort compared to provisioning cloud resources.
MSPs could help, but even with their assistance, the ongoing costs and complexity often favor cloud-native solutions.
How do you think we could convince small business owners to move from Shopify for example?
@cwebber At the moment we ended up with a phone that could do most of what we wanted, but had no agency over, at the backend there was the datacenter explosion, which we had no agency over. Now you can't even buy a laptop anymore because the datacenter explosion caused scarcity of its components. Check mate.
Working in that environment, seeing as Google rolled out the idea of "cloud computing" meaning "you have no involvement or agency in your computing because we do it for you" radicalized me for much of the work of my career.
It was one thing to run a datacenter to index the world's public web information. I understood that, it made sense.
But watching as Google and Apple co-developed the idea that computers, which I cared about, got abstracted into toys and jewelry that had all your key computing done in a way you had no agency over... where I saw firsthand the kinds of churn of resources necessary to keep these things going, it made me want to fight for a different computing future.
@cwebber The appliance-ization of computing really depresses and infuriates me.
Growing up with early personal computers in the 80s, the magic and the promise was you could use these machines to do *anything*, including things nobody else thought or cared to do!
A big part of what I fought to build in Android was a mobile platform that'd also deliver on the "you can run anything you want" promise of the machines I grew up with... sadly over the past decade that has been seriously eroded.
@cwebber
The core problem is not AI, cloud, or interfaces.
The real problem is ambiguous and centrally controlled information.
Without deterministic foundations, civilization will keep scaling dependency, manipulation, and chaos.
CTMinfo is the first real attempt to solve this at the root.
@cwebber do you have a writeup expanding on “datacenters are an anti-pattern” because this is 100% how I feel. I’m not fighting over AI. I’m still fighting over the cloud. Society has badly fumbled the fact that everyone has incredible computing power in their pocket. We’ve already been wasting electricity on idle servers and inefficient high level code. I’ve been - and am - part of the problem, moving functionality to the cloud for business reasons I hate.
@thomasjwebb I've been meaning to write a blogpost for a long time. Sounds like it's time to write it!
@cwebber @thomasjwebb I remember "datacenter" starting as "colocation hoster" - you rentet rackspace or several racks with redundant power supply, internet link, packed in some pizzaboxes and a router, and there you go. Physical safety was better than the rack with dev servers in the basement, so what else could you ask for?
@cwebber that's an interesting point of view. I mean, of course the current data center craze is complete madness, but it seems you consider an anti-pattern the concept of data center itself. Why is it so? What do you suggest as an alternate solution to the problems data centers try to solve?
@farfalk Datacenters are concentrations of power. Anytime a datacenter is involved, it's a sign of power centralization. The rise of datacenters corresponds with the death of p2p and other visions of a more decentralized internet.
@cwebber @farfalk Taking the long view, it’s the return of the data center coupled with public access and it is part of a swing between centralization and distribution of computers that dates to the 1960s.
The essential tension driving this is economy of scale versus localization of control. Before the PC it was departmental computing versus corporate control of resources.
I think data centers can serve a useful purpose but the pendulum has swung too far.
@cwebber @farfalk like, I can appreciate some of the advantages of having them. Like you could get more computer per watt, maybe. I think valuable research is done with supercomputers and modern, more modular approaches to big data. But we could do way more with way fewer datacenters if these weren't used as a way to paywall functionality at the server side. The move to the cloud almost makes me miss when my problem was Cubase requiring a USB dongle.
@cwebber @farfalk I think it more corresponds to the death of personal computing as it was? People don't have desktops anymore and barely have laptops other than for work? Which is a problem for p2p? Seems like most people's decentralized/federated nodes for things are hosted in data centers? All question marks because just speculating.
@johns @cwebber @farfalk it's telling that hardware for user-controlled computing is disappearing. Memory and storage are disappearing from the market and it feels *intentional*. https://investors.micron.com/news-releases/news-release-details/micron-announces-exit-crucial-consumer-business
Install bonfire.cafe
Get the full app experience
