Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
7 Boosts
0 Likes
@mattblaze Software is absolutely trustworthy. Stupid computer does precisely what it is told, every time. The problem of course is paying attention to what you are asking it to do.
@mattblaze for those saying “just an engineer saying….” There’s no way “just an engineer” is being quoted here. This is the way open AI want this portrayed.
For me, I think it’s a drive to normalise anything they don’t want to stand over as “rouge acts” and beyond their control. But they build and release these solutions and should be held accountable for _all_ its actions.
@mattblaze I mean a lot of the anti-AI messaging is that software is predictable and trustworthy so I guess it’s something both sides agree on
@mattblaze Any complex project has the capacity to run off the rails. "Going rogue" is a reasonable colloquial descriptive phrase even an engineer might use in passing.
It isn't meant to imply the project has a will of its own, only that it's no longer performing the expected purpose.
It's only in the era of fake AGI that we suddenly have to be so extremely careful with our wording. The general public has been conditioned to think this is all magic and demonology.
@gooba42 So the term is fine, but it’s the public’s fault for not understanding it? Uh huh.
@mattblaze No, I think policing the language that engineers use for a media who refuses to understand or follow up with clarifying questions is a problem.
We're facing this problem because the media has stopped caring whether they are translating accurately for their audience.
@gooba42 well, I’m an engineer and I don’t think “going rogue” is a useful term either colloquially or technically here. It’s buggy software. Or maybe not buggy, but intentionally malicious. But either way, software lacks the agency required to up and decide to “go rogue” on its own. It does what the software is programmed to do.
@mattblaze @gooba42 I'm not sure "programmed" conveys the entirety of how LLM based agentic systems are built. They can go beyond what they were told to do.
It's more like "they will do whatever is necessary to fulfill the task, unless explicitly programmed (sic) not to (and might not follow all the rules even then)".
This does not absolve the engineers, the enablers of responsibility. Just that the emergent behaviors exist and are broader than "programmed software".
@dominykas @gooba42 That’s just not how computers work. They execute instructions, which are encoded in software. The software might be hand assembled, generated by a complier, or synthesized from a machine learning system, but however the final executed instructions came about, a programmer (or a coven of programmers) was involved.
These AI systems didn’t crawl out of the ocean on their own. Someone built them, and their behavior reflects human decisions (or negligence).
@dominykas @gooba42 Now, that said, no matter how you’re creating software (from hand assembly to AI learning models or anything in between), ensuring correct, safe behavior is HARD. Evey programmer knows this (and there are fundamental reasons for it), which is why testing and debugging are an expected part of the development process, starting well before systems are inflicted on real users and consequential environments.
@dominykas @gooba42 Essentially what they're saying, implicitly or explicitly, amounts to: "The programming technique we've chosen to use absolves us of responsibility for our choices".
@mattblaze I've been writing code for production systems for 45+ years. When a program f*cked up, it was either due to an error I made during coding, or a business situation I hadn't anticipated.
I could follow the logic that caused the error and correct it.
But now with non-deterministic AI agents and neural networks that don't follow simple if..then..else logic to solve problems, it's almost impossible to determine the root cause until AFTERWARDS.
And somehow this is better. 🤬
@mattblaze Appreciate the gift article. Sadly, it seems that NYT gift articles now demand that you hand over your email address.
@verobeeee @mattblaze Pro tip: Often you don't have to put in a real address - fred at flintstone dot com is useful! There are a LOT of ways around paywalls (but if you use the workarounds a lot at any given site, you should probably pay up). https://dwightsilverman.com/2022/12/31/ads-overlays-and-paywalls-oh-my-how-to-get-around-the-webs-worst-annoyances/
@verobeeee Thank you for your feedback. The issue you mention is not somthing I can do anything about, because I don’t work for the New York Times. Perhaps you would have better luck addressing your concerns directly with them.
@mattblaze Oh, I wasn't expecting you to do anything about it. Just FYI. This is new.
@mattblaze Strong “the passive voice was used” energy.
@mattblaze "Rogue" also carries a huge amount of anthropomorphization with it. *Humans* go rogue. A rogue is rakish! Even lovable! Han Solo is a rogue. Everybody loves Han!
"Rogue" is a techbro euphemism for "we want you to think of this poorly programmed software as human so you'll engage with it more and more while we destroy your neighborhood with datacenters."
@mattblaze What if it was planned? (They say the posts were set on private etc ... that's very "human".). I don't believe those guys any word. And I'm not any cultish believer of dystopic pseudo-myths but a very curious journalist.
I hope that some good investigative journalists will dig deeper into this 💩
@mattblaze Shut them down. They are computer viruses.
@mattblaze at some point a programmer put in an option. “ if you don’t have valid credentials find valid ones” it was programmed to cheat, by someone. When VW programmed engines to know when the were being tested and decrease exhaust nobody said the engines went rouge.
@mattblaze
"...as if software is somehow usually trustworthy."
You're probably familiar with the UK Post Office Horizon scandal where some people assumed software is trustworthy. It didn't end well for them (and unfortunately for a lot of innocent people as well).
So a computer crime was committed.
There is a suspect.
Usually the police comes in and confiscates ALL computers that might have been used or possibly mightz be used for a complete crime. Like copying a disney song.
Similarly the police won't just raid and secure only the gun used in a murder, but all firfearms in possession of the suspect.
So why not at OpenAI, Anthropic or Google?
Then the threat will be eliminate, too.
Win-win!
@mattblaze it also suggests agency on the part of the software (look how smart it is!)- also as if the authors didn’t have anything to do with it
@mattblaze it also falsely implies software has volition, a claim the people reporting this are actively trying to push.