Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
10 Boosts
0 Likes
@mattblaze I think the problem is that while normal software can be trusted to do stuff that it's programmers have put in (however malicious that is), "going rogue" in relation to AI is supposed to make us think it's more sophisticated than it actually is. BUT it also tells people that AI is inherently untrustworthy.
@mattblaze "Bugs" don't generate giant stock overvaluations!
@mattblaze "Rogue" is just more anthropomorphizing of AI, which even if negative is still supporting the major tech narrative (in the sense of "no bad publicity"). It's lazy headline writing, and I wish it would stop. Well, I wish a lot of things would stop, most much more serious. But this is no way to cover negative consequences of a technology.
@mattblaze the MPR/AP article is even better. It engaged with a website. Just a little misbehavior...
https://www.mprnews.org/story/2026/09/26/npr-openai-us-government-websites-misbehavior
@mattblaze it also falsely implies software has volition, a claim the people reporting this are actively trying to push.
@mattblaze Software is absolutely trustworthy. Stupid computer does precisely what it is told, every time. The problem of course is paying attention to what you are asking it to do.
@mattblaze for those saying “just an engineer saying….” There’s no way “just an engineer” is being quoted here. This is the way open AI want this portrayed.
For me, I think it’s a drive to normalise anything they don’t want to stand over as “rouge acts” and beyond their control. But they build and release these solutions and should be held accountable for _all_ its actions.
@mattblaze I mean a lot of the anti-AI messaging is that software is predictable and trustworthy so I guess it’s something both sides agree on
@mattblaze Any complex project has the capacity to run off the rails. "Going rogue" is a reasonable colloquial descriptive phrase even an engineer might use in passing.
It isn't meant to imply the project has a will of its own, only that it's no longer performing the expected purpose.
It's only in the era of fake AGI that we suddenly have to be so extremely careful with our wording. The general public has been conditioned to think this is all magic and demonology.
@gooba42 So the term is fine, but it’s the public’s fault for not understanding it? Uh huh.
@mattblaze No, I think policing the language that engineers use for a media who refuses to understand or follow up with clarifying questions is a problem.
We're facing this problem because the media has stopped caring whether they are translating accurately for their audience.
@gooba42 well, I’m an engineer and I don’t think “going rogue” is a useful term either colloquially or technically here. It’s buggy software. Or maybe not buggy, but intentionally malicious. But either way, software lacks the agency required to up and decide to “go rogue” on its own. It does what the software is programmed to do.
@mattblaze I've been writing code for production systems for 45+ years. When a program f*cked up, it was either due to an error I made during coding, or a business situation I hadn't anticipated.
I could follow the logic that caused the error and correct it.
But now with non-deterministic AI agents and neural networks that don't follow simple if..then..else logic to solve problems, it's almost impossible to determine the root cause until AFTERWARDS.
And somehow this is better. 🤬
@mattblaze Appreciate the gift article. Sadly, it seems that NYT gift articles now demand that you hand over your email address.
@verobeeee @mattblaze Pro tip: Often you don't have to put in a real address - fred at flintstone dot com is useful! There are a LOT of ways around paywalls (but if you use the workarounds a lot at any given site, you should probably pay up). https://dwightsilverman.com/2022/12/31/ads-overlays-and-paywalls-oh-my-how-to-get-around-the-webs-worst-annoyances/
@verobeeee Thank you for your feedback. The issue you mention is not somthing I can do anything about, because I don’t work for the New York Times. Perhaps you would have better luck addressing your concerns directly with them.
@mattblaze Oh, I wasn't expecting you to do anything about it. Just FYI. This is new.
@mattblaze Strong “the passive voice was used” energy.
@mattblaze "Rogue" also carries a huge amount of anthropomorphization with it. *Humans* go rogue. A rogue is rakish! Even lovable! Han Solo is a rogue. Everybody loves Han!
"Rogue" is a techbro euphemism for "we want you to think of this poorly programmed software as human so you'll engage with it more and more while we destroy your neighborhood with datacenters."
@mattblaze What if it was planned? (They say the posts were set on private etc ... that's very "human".). I don't believe those guys any word. And I'm not any cultish believer of dystopic pseudo-myths but a very curious journalist.
I hope that some good investigative journalists will dig deeper into this 💩
“Our software went rogue!” No. You programmed it wrong.
If you don’t want your system to be able to lock people out of the pod bay doors, you can just program or configure it to not have the capability to lock people out of the pod bay doors. This remains true whether your system is written in assembly language or C++ or Python or uses some fancy machine learning model.
@mattblaze I think you’re mixing up deterministic and non deterministic software here.
Before AI, software was programmed to run from A to B or to discover and react to events deterministically. Even the bugs were deterministic.
After AI, agents were programmed to read code, write code, and run code. You can sandbox them, only give them access to certain tools, but to be useful, they really do need to run tools and code you haven’t thought of yet and therein lies the dilemma.
@kumarvibe I suspect by “non-deterministic” you actually mean “randomized”. “Non-deterministic” involves a computational model of calculating results in parallel.
But randomized algorithms and software are not new at all. Plenty of systems use randomization. None of that changes the fact that this is all just software, created and controlled by humans, running on computers with capabilities that are also created and controlled by humans.
@mattblaze yes, models are human controlled for sure, but whenever I think about the killing problem I’m not sure how else you could solve it aside from programming ethics into the model. There are just too many ways to kill 😂
You could, of course, limit access to all the killing things but then you’re left with no internet access and so on — too many limitations to be of use, probably 🤷
In any case, every “rogue” agent attack involved a sandbox breakout. Building the sandbox is not easy.
@kumarvibe Software correctness and safety isn’t easy. Who told you it was?
But the killing problem can be solved. My word processor can’t kill anyone (Although maybe it could raise my blood pressure by deleting files at an inopportune moment).
Perhaps if you aren’t sure what a piece of software might do, you shouldn’t let it control lethal capabilities?
@mattblaze @kumarvibe I have sixty years in the software industry and I endorse this statement.
Greed is what’s pushing this forward.
@holdenweb oh hey Steve, good to see you on here. Big respect for all your work in the Python community.
@mattblaze unlike word processors, it would only take a minute for a “rogue” agent to break out of its sandbox and trick Donald Trump into annihilating the planet with nukes. I am not sure how one would fix that problem aside from training agents on ethics.
And, yes, it’s fueled by greed but Pandora’s box is open. There is no going back.
@kumarvibe @holdenweb Huh? Don’t give software with undefined behavior, whether it uses “AI” or not, access to capabilities that you don’t want it to misuse. Ever. Period. That’s the trick. (Actually it’s not a trick, it’s just an obvious safety precaution.)
You either put it in a configuration with access to the Internet (or the life support system or whatever) or you don’t. Software doesn’t “escape” containment. It’s that someone failed to contain it in the first place.
@kumarvibe @holdenweb You seem to be suggesting that giving random software (or “AI agents” or whatever it’s called this week) with undefined behavior access to dangerous capabilities is somehow necessary for society to be able to function efficiently.
I think that’s ridiculous and indefensible. The fact that something has “AI” pixie dust doesn’t somehow cancel out basic principles of safety engineering.
@mattblaze @kumarvibe @holdenweb Hundreds of billions of dollars spent by these so-called geniuses and not one of them knows how to unplug an Ethernet cable.
@mattblaze
Conspicuously, “went rogue” is an example of the past exonerative, up there with “shots were fired”, “the gun went off”, “the car lost control”, “the market lost billions”, etc.
Use of the past exonerative tense should always make the reader at least a little suspicious.