Post
What is the incremental environmental damage created by running an existing LLM locally on your own laptop?
As to "90% bullshit" - as I wrote, the false positive rate for punctuation errors and typos from Ollama/Llama2 is about 50%, which is substantially better than, say, Google Docs' grammar checker.
@tante Use of the word "neoliberal" in earnest in an essay is an almost infallible sign of nonsense. Certainly it works here.
@tante Thanks for putting this into words. Struck me the same way.
@tante “Technology doesn’t need liberation, people do.“ It seems so crystal clear to me.
That doesn't seem to be the best idea @pluralistic
AI and LLM output is 90% bullshit, and most people don't have the time nor the patience to work out which 10% might actually be useful.
That's completely ignoring the environmental and human impacts of the AI bubble.
Try buying DDR memory, a GPU or an SSD / HDD at the moment.
@simonzerafa @tante @pluralistic
At best 40% junk, but unless you are so expert you don't need it, you can't know which is plausible rubbish.
Would you play Russian Roulette every day for hours?
Beats me.
I thought Cory was supposed to be clever or something? I've blocked him for now. Not interested in banging my head against that particular lack of critical thinking.
Perhaps when the AI bubble bursts, he will become more rational.
@simonzerafa @raymaccarthy this is just a dumb attack on Cory as a person which I will not accept. You can talk about what he or I wrote (both things can be criticized) but have some respect
@tante @simonzerafa
A brilliant person isn't right about everything.
It's only a criticism of one view/idea.
@raymaccarthy @simonzerafa @tante
Again, what does checking the punctuation on a single essay per day have to do with "play[ing] Russian Roulette every day for hours?"
What is the incremental environmental damage created by running an existing LLM locally on your own laptop?
As to "90% bullshit" - as I wrote, the false positive rate for punctuation errors and typos from Ollama/Llama2 is about 50%, which is substantially better than, say, Google Docs' grammar checker.
@pluralistic @simonzerafa @tante I hate to dive into what is clearly a heated debate, but I want to add an answer to your question with a perspective that I think is missing: the power consumption for inference on your laptop is probably greater than in a datacenter. The latter is heavily incentivized to optimize power usage.
parsing a doc uses as much juice as streaming a Youtube video and less juice than performing a gnarly transform on a hi-rez in the Gimp.
I measured.
@pluralistic @simonzerafa @tante
"What is the incremental environmental damage created by running an existing LLM locally on your own laptop?"
I dunno. But how about a couple of million people?
The person who coins the term 'enshittification' defends LLM. Just...wow. We truly are fucked.
Let's all do what Cory does!
☠️
Meanwhile:
https://www.technologyreview.com/2025/05/20/1116327/ai-energy-usage-climate-footprint-big-tech/?gad_source=1&gad_campaignid=20737314952&gbraid=0AAAAADgO_miNIDzn-BdCIXzZ6r87g94-L&gclid=Cj0KCQiA49XMBhDRARIsAOOKJHbvIzPACe0EdEyWK86TnS7rNlnUaePKc5y22qT0ZsfqUeGDe72zzc0aAhFFEALw_wcB
#doomed #ClimateChange
@clintruin @simonzerafa @tante
Which "couple million people" suffer harm when I run a model on my laptop?
@pluralistic @simonzerafa @tante
Missed the point, sir.
When one person does it...no big deal.
When a couple of million people do it...well, see the MIT article above.
@pluralistic @simonzerafa @tante
Subhead quote from the article:
"The emissions from individual AI text, image, and video queries seem small—until you add up what the industry isn’t tracking and consider where it’s heading next."
@clintruin @simonzerafa @tante
You are laboring under a misapprehension.
I will reiterate my question, with all caps for emphasis.
Which "couple million people" suffer harm when I run a model ON MY LAPTOP?
@pluralistic @clintruin @simonzerafa @tante
Which "couple million people" suffer harm when I run a model ON MY LAPTOP?
Anyone who's hosting a website, and is getting hammered by the bots that seek content to train the models on. Those of us are the ones who continue getting hurt.
Whether you run it locally or not, makes little difference. The models were trained, and training very likely involved scraping, and that continues to be a problem to this day. Not because of ethical concerns, but technical ones: a constant 100req/sec 24/7, with over 2.5k req/sec waves may sound little in this day and age, but at around 2.5k req/sec (sustained for about a week!), my cheap VPS's two vCPUs are bogged down trying to deal with all the TLS handshakes, let alone serving anything.
That is a cost many seem to forget. It costs bandwidth, CPU, and human effort to keep things online under the crawler DDoS - which often will require cold, hard cash too, to survive.
Ask Codeberg or LWN how they fare under crawler load, and imagine someone who just wants to have their stuff online having to deal with similar abuse.
That is the suffering you enable when using any LLM model, even locally.
@pluralistic @simonzerafa @tante
I'll reiterate my response.
When you *alone* do it...no big deal.
When a couple of million do it ON THEIR OWN LAPTOPS...problem.
@clintruin @simonzerafa @tante
OK, sorry, i was under the impression that I was having a discussion with someone who understands this issue.
You are completely, empirically, technically wrong.
Checking the punctuation on a document on your laptop uses less electricity than watching a Youtube video.
@pluralistic @simonzerafa @tante
Fair enough, Cory. You're gonna do what you want regardless of my accuracy or inaccuracy anyway. And maybe I've misunderstood this. The same way many many will.
But visualize this:
"Hey...I just read Cory Doctrow uses an LLM to check his writing."
"Really?"
"Yeah, it's true."
"Cool, maybe what I've read about ChatGPT is wrong too..."
@clintruin @simonzerafa @tante
This is an absurd argument.
"I just read about a thing that is fine, but I wasn't paying close attention, so maybe something bad is good?"
Come.
On.
@pluralistic @simonzerafa @tante
Maybe...
Maybe not.
You have a good day.
@pluralistic @simonzerafa @tante
But hey, you do you, Cory.
I'm nobody...your Cory Doctrow.
Let's all do what Cory does...
@clintruin @simonzerafa @tante
Well, you could "do what Cory does" by familiarizing yourself with the conduct that you are criticizing before engaging in ad hominem.
To be fair, that's not unique to me, but people who fail to rise to that standard are doing themselves and others no good.
@pluralistic @simonzerafa @tante
But Google Docs anything is rubbish.
@raymaccarthy @simonzerafa @tante
I see. And do you have moral opinions about whether people should use Google Docs? Do you seek out strangers to tell them that it's dangerous to use Google Docs?
Of course, I am speaking in generalities.
Encouraging the use of LLM's is counterproductive in so many ways, as I highlighted.
Pop a power meter on that LLM adorned PC and let us all know what the power usage looks like with and without your chosen LLM running on a typical task 🙂
That's power that generated somewhere, even if it's with renewable energy.
The main issue with LLM's is that they don't encourage critical thinking, in a world which is already suffering from a massive shortage.
As I wrote (and it seems you haven't read what I wrote, which is weird, because that seems like a good first step if you're going to criticize my conduct), I'm running Ollama on a laptop that doesn't even have a GPU.
Its power consumption is comparable to, say, watching a Youtube video.
I know this because my laptop is running free software that lets me accurately monitor its activity, and because the model is also free software.
Checking for punctuation errors is does not discourage critical thinking. It's weird to laud "critical thinking" and also make this claim.
@pluralistic @simonzerafa on this one for example I fully agree with Cory. This is not him having a genAI system write or anything like that.
@tante @pluralistic @simonzerafa I agree in principle with Cory, but I really wish that he had clarified that:
1. Ollama is not an LLM, it's a server for various models, of varying degrees of openness.
2. Open weights is not open source, the model is still a black box. We should support projects like OLMO, which are completely open, down to the training data set and checkpoints.
3. It's quite difficult to "seize that technology" without using Someone Else's Computer to do so (a.k.a clown/cloud)
@tante @pluralistic @simonzerafa But ALSO: using a multi-billion-parameter synthetic text extruding machine to find spelling and syntax errors is a blatant example of "doing everything the least efficient way possible" and that's why we are living on an overheating planet buried under toxic e-waste.
If I think about it harder I could probably come up with a more clever metaphor than killing a mosquito with a flamethrower, but you get the idea.
No. It's like killing a mosquito with a bug zapper whose history includes thousands of years of metallurgy, hundreds of years of electrical engineering, and decades of plastics manufacture.
There is literally no contemporary manufactured good that doesn't sit atop a vast mountain of extraneous (to that purpose) labor, energy expenditure and capital.
@pluralistic @dhd6 @tante @simonzerafa
The patent system is designed around an acknowledgement that we invent while "standing on the shoulders" of those who have gone before -- this premise is built into the patent system; policy decisions support this approach.
Patent grants exclusive rights to the inventor for a limited time. In exchange, the.inventor must disclose how a person versed in the arts can replicate that invention.
The same is not true of literature / copyright. Different animal, different approach.
@pluralistic @tante @simonzerafa As always, yes and no. A bug zapper is designed to zap bugs, it is a simple mechanism that does that one thing, and does it well. An LLM is designed to read text and generate more text.
That we have decided that the best way to do NLP is to use massively overparameterized word predictors that we have trained using RL to respond to prompts, rather than just, like, doing NLP, is just crazy from an engineering standpoint.
Rube Goldberg is spinning in his grave!
Remember when Usenet's backbone cabal worried about someone in Congress discovering that the giant, packet-switched research network that had been constructed at enormous public expense was being used for idle chit chat?
The nature of general purpose technologies is that they will be used for lots of purposes.
@pluralistic @tante @simonzerafa indeed, I guess the question is whether the scale of the *ahem* waste, fraud and abuse *ahem* of resources that LLMs seem to imply, even in benign use cases like yours, is out of line with historical precedent or not.
Am I an old man yelling at a cloud?
No, it's the children who are wrong!
Rockets were literally perfected in Nazi slave labor camps.
@pluralistic @dhd6 @tante @simonzerafa Good grief, these ad hoc rationalizations are absurd and you know it.
FYI, rockets are enormously environmentally destructive (fuel, pollution, noise, etc.). The planet would be better off with as few rockets launching as possible.
Saying an LLM is OK because some completely other "good" technology was invented by evil people is a *non argument*.
@jaredwhite @dhd6 @tante @simonzerafa You're right, that would be a silly thing to say.
Good thing I didn't say it.
@pluralistic @dhd6 @tante @simonzerafa what a shit take dude. rockets being perfected by nazis, project paperclip, and now a neonazi in charge of one of the largest space tech programs on the planet, along with a bullshit generating LLM.
so yeah, maybe this is all fash tech, and maybe taking a stand of "I'm not touching that shit with a thousand-meter pole" is not "neoliberal purity culture". and ollama of all things? the shit pumped out by fucking Meta? are you shitting me?
@elle @dhd6 @tante @simonzerafa
"You used the wrong open model because I don't like the company that made it" is the actual definition of nonsense purity culture.
@pluralistic @elle @dhd6 @tante @simonzerafa I beg to differ. Demand is a powerful and legit tool of people responding to corporate behavior. Choosing a different product because you dislike a maker's conduct is nought but the invisible hand of the market slapping that maker for their conduct. Provenience does matter.
Smearing choice of provenience as wilful purism would be the perfect argument for any company to disregard social or ethical standards, constituting a right to demand for the 'best' offer. Which would take us into the field of classic objectivism, and be in itself as willful and naive as the purism it accuses consumers of.
@pluralistic @dhd6 @tante @simonzerafa you wrote a book on how much of a shitbag company corpos like Meta are. now you're saying "oh it's not that bad, look it's marginally better than Google Docs spell checker"?! did someone hack your fucking account?
there are legitimately open models that originate from academic institutions, train on open data with full consent. even those models take tens-of-thousands of euros to train. well outside the resources available to most open-source enjoyers
I am astonished that I have to explain this,
but very simply in words even a small child could understand:
using these products *creates further demand*
- surely you know this?
Well, either you know this and are being facetious, or you are a lot stupider than I ever thought possible for someone with your privilege and resources.
I am absolutely floored at this reveal, just wow, "where's Cory and what have you done with him?" 🤷
Massive loss of respect!
@tante There must be something in the air; I think this is something that has been on many folks' minds for a while... not criticisms of Cory, but rather the general cognitive difficulty of reconciling survival in a world far less under our control than we thought it was when our character and values accreted and congealed.
It's a nuanced phenomenon especially for a limited-attention-span world. It's good to see people taking up the challenge. Thanks for this.
Install bonfire.cafe
Get the full app experience
