Discussion
A phenomenon I've noticed recently is people trying to occupy some untenable middleground wrt to the use of systems sold as "AI" -- this is a position where people try to recognize the harms of this tech but also hold space for "responsible" or "ethical" use.
🧵>>
@emilymbender The argument is compelling. Now where are the "yeah but" brigade? I fancy entering the fray but I might need some training, and where better than here.
Come on you cowards, tell us how "inevitable" it is, give us your best shot.
@emilymbender Thank you so much for this thread Emily 💚 It's so good, I'm considering printing a paper copy to keep in my pockets for the next time I inevitably encounter theses arguments in the wild again.
Please let us know if you decide later to store this thread in an article form online. I would share this link with everyone!
@emilymbender Thank you so much for this thread Emily 💚 It's so good, I'm considering printing a paper copy to keep in my pockets for the next time I inevitably encounter these arguments in the wild again.
Please let us know if you decide later to store this thread in an article form online. I would share this link with everyone!
@emilymbender why is that untenable? i can see ecological or social reason to choose otherwise, but it seems rational to use tools when helpful even if they are dangerous when abused or otherwise misused (e.g. battery chainsaws, electric cars, etc.)
When someone is trying to hold this untenable position, a few things tend to come up (not everytime, not everyone):
1- Defensiveness. People read criticism of the systems and proposed uses of the systems as accusations that users are "bad people". Thus a criticism of the tech lands as criticism of the user, and tensions flare.
>>
2- Righteousness. People do have legitimate needs, often unmet needs, and the synthetic text extruding machines can *look like* a solution. But just because the problems are real doesn't mean the solution is beneficial, effective, or worth (not always externalized) costs. Unfortunately, pointing out any of this is taken as the same as saying you don't care about the legitimate needs.
>>
@emilymbender This is actually the strongest one. An ESL job seeker in America is going to absolutely get creamed in the market, especially right now, when their imperfect English marks them as a no-hire. It's perfectly reasonable from their perspective to run their C.V. and cover letters through an LLMbecile to paper over the spelling, grammar, and improper idiom use cases.
Of course the real solution is for hiring managers not to be judgmental pricks, but that doesn't help the immediate need.
@emilymbender this was key in helping talk to a friend about their LLM use. Acknowledging the need and lack of social support and offering real human connection and aid instead helps them feel emotionally safe and heard. Then they're open to listening to the moral theme, because their needs have been acknowledged or met.
3- Whataboutism. This is used to brush off concerns about the externtalities of these systems. You eat meat, you fly on airplanes, etc, etc, how dare you talk about the impacts of data centers?
>>
@emilymbender
Makes sense to me for 2 reasons:
(a) if, in the 1920s, if a few folks had truly understood the Geography of Nowhere & environmental impact of widespread automobile usage, they'd have sought to prevent it by any means necessary (regardless of other bad acts going on).
(b) There is so little hard data (even projections) available that shows us how bad it's going to get vs. past environmental disasters.
I really wish we had (b) as settling the Whataboutism argument would be trivial.
@emilymbender I like to call the people who do this gotcha guys after this Matt Bors comic https://thenib.com/mister-gotcha/
4- Tone policing. People who are trying to occupy that uncomfortable, untenable space will claim that clear statements of harms/strong principles against use of these systems will "turn others away" as if the centrists are the ones actually pushing for more ethical practice.
But this "other people won't listen" remark I think is really a way of saying "This makes me uncomfortable" while trying to claim to be on the right side of history at the same time.
>>
@emilymbender I have met several examples of this in tense political discussion on various subject, this needs a specific word I guess. Maybe it has?
5- Wishcasting. Some folks will point to scientific results from fields outside their own (usually media coverage thereof) that are marketed as having been done with "AI" and ask: How could you take a hardline against "AI" when it has provided XYZ?
>>
6- Exceptionalism. "I know this can be dangerous for people in general, but I know how to use it carefully."/"I know how to verify every output, and I am not deskilling myself." How do you know? Also, if you acknowledge the dangers to others, what example are you setting by talking about/talking up your use?
>>
@emilymbender my favorite part of the exceptionalism is the implication that one has finished learning. and the sheer arrogance of believing they are immune to their psychological trickery
So what is the best way out of that uncomfortable, untenable space? I think one key step is disaggregating the (non-coherent) set of technologies sold as "AI". If you don't call the stuff you work with "AI", you aren't saddled with trying to defend any of the rest of it.
>>
Thanks for this thread. I'm particularly interested in your examples of why AI it is not a useful technical term (and we should be referring to specific tools instead) especially within the context of critiques of AI as a marketing term.
I'm collecting resources on this topic (for folk not on Mastodon). Anything you'd recommend?
The most recent iteration of this conversation I was involved in turned in part on a strange, over-expansive definition of "genAI" which included, for ex, optical character recognition (OCR).
>>
@emilymbender Eugh, I’m seeing CGI effects on movies being called AI now, even when it predates genAI.
If the kids use “AI” as synonymous with “slop” and “photoshopping”, do we have to list it in the dictionary as that?
But this is literally no different to when analog VFX started being called CGI even though they clearly used no computers.
@emilymbender Same goes for automation. We had scripts before AI, we'll still have scripts after AI.
I don't get the point of connecting AI to all needs.
OCR can be a useful tool for many research projects! OCR is also the kind of technology that gets better with better language models, i.e. more fine-grained models of which word(parts) go where. That has been true since before "genAI" and will be true after.
Just because you can use the synthetic media extruding machines to approximate the task of OCR, however, doesn't mean that that task can or should be used to justify the use of "genAI" in research.
>>
@emilymbender So even you yourself strike a middle ground - OCR is OK in academic scenarios. Noted.
@emilymbender
OCR = Optical character recognition?
@emilymbender
Thank YOU, Prof Bender.
I'm enjoying your book "The Ai Con," now available on Bookshop.org.
@emilymbender Is neural machine translation generative AI?
@emilymbender
OCR is about 45 or 50 years old and gradually getting better. About 40 years since it was independent of font?
The LLM/Generative AI is nothing much to do with other technologies that traditionally have had an AI label. It's a dead end.
@raymaccarthy Stay 'splainy mastodon.
@emilymbender Pretty sure the 'splaining is a human thing, not a Mastodon thing.
@emilymbender Pretty sure the splaining is a human thing, not a Mastodon thing.
I think another important step is a values examination. What is important to you? How are those values supported or not by entering the discourse in a way that holds space for OpenAI/Anthropic/Google/Meta and all the other actors in this massive push to shove "AI" into every part of our lives as "not all bad"?
What are your research goals, what do you value about participating in scholarship, how can you meet those goals/act in accordance with those values and what obstacles are in your way?
>>
Part of what makes that middle ground untenable and uncomfortable, I think, is that it requires carrying water for these clearly bad actors. You can set that bucket down and step out onto firmer ground.
This does require going against the mainstream, but that gets easier when a) you find you're not alone and b) you see how much of mainstream opinion on this is actually the result of marketing.
/fin for now
@emilymbender
Middleground guy here.
Of all proposed uses for LLMs, I found one which is really useful. Cybersecurity.
Fuzzing is a method to inject random data into a program to make it crash and so discover security vulnerabilities.
Nowadays LLMs are used to create precisely targeted data to inject. They go through the source code to determine what to inject.
So insted of throwing random data at the problem for days, this method is way faster.
Shall we name this something specific?
The frustrating thing for me is how much of "old fashioned" machine learning gets lumped in to "AI" conversations these days, and then its (very real) usefulness gets used to prove the value of "AI," which is then used as an umbrella defense for setting LLMs on bullshit extrusion mode.
Even language models—possibly even large ones!—are very useful for translation, transcription, captioning, first passes of text annotation. i.e., jobs that involve interpreting language as language and doing language things with them.***
As you say, it's the generative nature of them—asking them to create or respond with meaning that wasn't in the input text itself—that starts the spiral out into resource-sucking noise.
(***maybe this sounds like the "middle ground" you're decrying but I think drawing a hard line at novel generation cuts off most of the hype nonsense.)
I'll add to this, in re: Dr. Bender's last post about bad actors:
Once you eliminate the generative cases, you also eliminate a huge amount of the demand for GPUs, power, and data centers. You remove the illusion that these models can replace workers. You cut off the ability to use models to effectively steal work.
In other words, you undercut a lot of the capitalist justification for titanic dumps of cash into the industry. Which is a good thing.
The frustrating thing for me is how much of "old fashioned" machine learning gets lumped in to "AI" conversations these days, and then its (very real) usefulness gets used to prove the value of "AI," which is then used as an umbrella defense for setting LLMs on bullshit extrusion mode.
Even language models—possibly even large ones!—are very useful for translation, transcription, captioning, first passes of text annotation. i.e., jobs that involve interpreting language and language and doing language things with them.***
As you say, it's the generative nature of them—asking them to create or respond with meaning that wasn't in the input text itself—that starts the spiral out into resource-sucking noise.
(***maybe this sounds like the "middle ground" you're decrying but I think drawing a hard line at novel generation cuts off most of the hype nonsense.)
This totally tracks.
In cyber incident response (training), I’ve been using the “five stages of grief”, or the Kübler-Ross model, to help incident responders deal with and understand people’s emotional reactions.
People understandably get upset when things they’ve invested ego in break down, are gone or they have to let go.
Anyway, the five emotions are (not necessarily in this order):
https://en.wikipedia.org/wiki/Five_stages_of_grief
PS: The Wikipedia page says “Though widely used, the theory is empirically unsupported.”
For what it is worth, it is empirically supported by me, from direct experience, after coordinating many, many cyber incidents.
@emilymbender Spot on! I read the first few posts in the thread and was like "yes, but too many things are called AI today that are useful, where the bad stuff is all tied to that name" and then you untangled that exact point. Perfect, no notes.
@emilymbender any thoughts on @pluralistic 's take on his limited use of AI?
https://www.theatlantic.com/podcasts/2026/06/how-to-think-about-ai-before-its-too-late/687644/
@emilymbender @pluralistic thanks for reminding me of this thread, where you set out the ethical issues clearly. Re-reading I note I did I actually contribute a post to it.
@emilymbender So are you saying there is no possibility for the ethical implementation of AI? I have argued against the use of AI in my own community, that of visually impaired people, and, not without some justification, have taken a lot of heat for doing so. AI has enabled visually impaired people to address issues that no-one else will for themselves such as software and web accessibility; it has facilitated greater independence for them in terms of accessing text.
@JustinMac84 @emilymbender
I feel like Professor Bender's thread answers that? From her thread:
> OCR can be a useful tool for many research projects! OCR is also the kind of technology that gets better with better language models, i.e. more fine-grained models of which word(parts) go where. That has been true since before "genAI" and will be true after.
> Just because you can use the synthetic media extruding machines to approximate the task of OCR, however, doesn't mean that that task can or should be used to justify the use of "genAI" in research.
OCR helps visually impaired people by extracting text from an image, so it can make things more accessible. But we used to just put it under the "computer vision" area of research, not the jumble of "AI" that only helps the tech giants sell their extremely harmful products. It instantly becomes "oh so you're against better tech for visually impaired people/better cancer screening/whatever?". It turns it into a bad faith argument. There's more than one way to make tech accessible for everyone. I think it's worth it to push back on the fact the only time tech giants can be arsed to make things more accessible is by implementing it through some of the most environmentally and socially damaging ways possible.
@igimenezblb @emilymbender Visually impaired people are not research projects, neither are they necessarily involved in scholarship and therefore I feel the application is different. As alluded to in my previous posts, AI-based OCR appears better than predecessors I've used and, crucially, the systems will instruct you on how to improve the picture taken, pointing out defects, where the text is, where it is cut off, blurry or subject to glare etc.
@igimenezblb @emilymbender Also, AI-based systems are able to offer instructions for assembly, source information based on model numbers etc, which greatly improves VI independence. Please don't misunderstand me, I'm far from an AI apologist as my followers and those who have unfollowed me through weariness or offense at my broadly anti-AI position would attest, but is an absolutist position realistic or justified given the gaps it does fill?
I am not against tech for supporting accessibility (when it is actually co-designed with the users it's meant for!). I understand that image-to-text systems, OCR, etc can be important for visually impaired people. Such systems could also be built without most or all of the externalities of the models being pushed by the big tech companies.
>>
Trouble is, I don't think right-sized projects, actually focused on the needs of visually impaired people are getting sufficient funding. So often, the only (or most convenient or...) tool to hand is one of these highly compromised systems.
>>
I have no beef with a visually impaired person using them in that case.
However: that doesn't mean "genAI" has ethical use cases. It means someone made the best decision available, and we should fix things so that better ones become possible, while not letting the tech companies appropriate the experience of people with disabilities to burnish their (the companies') reputations.
>>
@emilymbender @JustinMac84 @igimenezblb Hi Prof. Bender. Blind senior data/AI lawyer here. I am also fundamentally opposed to gen AI. This is a good take overall. You are correct that accessibility projects are underfunded, and that we'd all vastly prefer having tech that works for us. I detest these solutions. But if a vibe-coded app means I can continue to do an asppect of my job, I have to use it. I think this is an important coralary
@emilymbender @igimenezblb Aren't the primary problems with AI its environmental impact and the exploitative uses to which big tech wants to put it at every level, in data harvesting methodology, the concentration of wealth and the theft of intellectual proprties. If we could make data centres green and regulate the CEOs, don't the problems go away. AKA isn't it the implementation that is the problem, rather than the tech itself?
@emilymbender @igimenezblb What about a VI person, or some other disabled/poor/marginalised person using AI to code something that improves their lives and, most importantly, the lives of those around them. Someone with whom I often have pro-anti AI debates posted an example the other week I wish I could find, something about an inaccessible restaurant website and she AIed a thing so that she and her party could circumvent the inaccessibility.
@JustinMac84 Describe this scenario with the same kind of language I was using and you will see the answer. Call it "AIing a thing" and you are back to carrying water.
@emilymbender I wish the character limit didn't completely preclude block quoting because then I could demonstrate that I am meaningfully engaging with your posts. What language do you think would be less problematic for me to describe the scenario I did the? Vibe coding is no less big tech enabling than "AIing", but it accurately describes what was done. I can't think of a non-water-carrying term.
Here:
Inaccessible websites are real problem for visually impaired people. Societally, we aren't putting enough resources into handling that problem, both in terms of enforcing/applying the ADA and in making it straightforward to create accessible websites.
>>
@JustinMac84 Some enterprising VI people have been able to hack workarounds, using text-to-code systems that work well enough for them to want to keep using them, and these workarounds are ones they wouldn't be able to do (or do easily enough) without such systems.
>>
@JustinMac84 While we could imagine creating text-to-code systems that are free from many of the externalities of the big models (unethical data collection, labor exploitation, massive environmental impact, societal impacts of marketing them as replacements for professional software engineering, etc), these have not been created and/or the person in question doesn't know how to find them if they do exist.
>>
@JustinMac84 In this case, it is understandable why the person would use the big (bad) models for text-to-code generation. They can talk about that use case while still acknowledging the harms of the big models, and without using it to ethics-wash the creation of those big models.
>>
@JustinMac84 I really recommend @FractalEcho 's book for deep background on how the experiences of people with disabilities are appropriate for the aims of Big Tech:
@emilymbender I am pretty sure I understand everything you were kind enough to write in those last posts and thank you for it. My issue and confusion has been that you wrote: "this is a position where people try to recognize the harms of this tech but also hold space for "responsible" or "ethical" use."
@emilymbender In your last thread to me you wrote: "In this case, it is understandable why the person would use the big (bad) models for text-to-code generation. They can talk about that use case while still acknowledging the harms of the big models, and without using it to ethics-wash the creation of those big models."
@emilymbender If I understand you correctly then, you are saying that, in those circumstances, that position would be tenable, in which case we agree and, indeed, it wouldn't be a problem if we didn't, I was just struggling to see how, based on your first post, the position could be held to be totally invalid. As I say, thank you for taking the time to engage with my arguments. I appreciate it.
@JustinMac84 My point is that trying to say that there is responsible or ethical use of "AI" doesn't hold enough nuance.
* The large models are always harmful because of how they are created, at the very least. (And can be used in ways that are more harmful.)
* Some things called "AI" aren't those, and talking about them as if they were the same thing leads to ethics washing.
>>
* Sometimes people are not afforded the choices they should be, and turn to the harmful models as the least-worst option.
* Using those people's experiences to argue for "ethical use" of inherently harmful tech both hides the harms that have left people in those corners and uses their experiences to do further ethics washing.
>>
@JustinMac84 I think people have a really hard time reasoning about this stuff because "unethical" reads as "bad person" and people say "But they can't be a bad person, they are just trying to do what is needed in the absence of accessibility/resources/etc."
