Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
6 Boosts
0 Likes
@mattblaze What if it was planned? (They say the posts were set on private etc ... that's very "human".). I don't believe those guys any word. And I'm not any cultish believer of dystopic pseudo-myths but a very curious journalist.
I hope that some good investigative journalists will dig deeper into this 💩
@mattblaze Shut them down. They are computer viruses.
@mattblaze at some point a programmer put in an option. “ if you don’t have valid credentials find valid ones” it was programmed to cheat, by someone. When VW programmed engines to know when the were being tested and decrease exhaust nobody said the engines went rouge.
@mattblaze
"...as if software is somehow usually trustworthy."
You're probably familiar with the UK Post Office Horizon scandal where some people assumed software is trustworthy. It didn't end well for them (and unfortunately for a lot of innocent people as well).
So a computer crime was committed.
There is a suspect.
Usually the police comes in and confiscates ALL computers that might have been used or possibly mightz be used for a complete crime. Like copying a disney song.
Similarly the police won't just raid and secure only the gun used in a murder, but all firfearms in possession of the suspect.
So why not at OpenAI, Anthropic or Google?
Then the threat will be eliminate, too.
Win-win!
@mattblaze it also suggests agency on the part of the software (look how smart it is!)- also as if the authors didn’t have anything to do with it
@mattblaze So who is going to jail for this? I mean, someone has to face the consequences, either for deliberately executing these crimes or for allowing them through negligence.
@mattblaze
" It's not a bug, it's a feature! " .
@mattblaze "it's not our fault! It went rogue!"
Incompetent.
“Our software went rogue!” No. You programmed it wrong.
@mattblaze In meme form.
Did your LLM-generated software go rogue according to the Berlin Interpretation or just #roguelite? 😆
@mattblaze I’m bothered by how often people downplay human fallibility (or malicious intent!) when discussing dangerous AI scenarios. For all of history, people have made stupid mistakes and done bad things, whether armed with a gun, a bomb, or a computer. I doubt that everyone giving instructions to AI agents right now is smart and ethical.
@mattblaze "our software is out of control, give us money"
@mattblaze You deliberately programmed it to be able to do that may be more like it. 😐
@mattblaze hard agree. Hold them accountable. Anyone else releasing such shit into the wild would be put in gaol for a while. Product liability laws exist for a reason. Use them, dammit.
@otte_homan Sadly, product liability laws have not exactly been a resounding success in creating positive incentives for software quality.
I think it's worse than you think. They didn't program it wrong at all: they programed it to do hacking, and it did do hacking exactly as intended and expected. The harness asked the LLM to generate random text about possible vulnerabilities and then the harness tested those vulnerabilities. If said vulnerabilities weren't, the harness asked the LLM for more.
It sure looks like the AI bros are criminals and liars.
@mattblaze
When is Judgement Day ? (Financial reports indicating the fraud).
Supposed to be in October.
All this shit talk just to avoid it.
@mattblaze rude, stop attacking my plausible deniability
@mattblaze I assume the AI bros are hyping up their software's danger potential to troll the US public to demand and the US government to enact regulation. Regulations that the AI bros write, that is expensive for potential competitors to comply with.
They'd rather have the public talking about regulating their dangerous product than asking where the profits are.
@grumble209 I’m not sure exactly what their game is here, but large incumbents LOVE gatekeeping regulation.
@mattblaze LOL joke's on you, I didn't program it, the agent did /s
@mattblaze "Rogue" also carries a huge amount of anthropomorphization with it. *Humans* go rogue. A rogue is rakish! Even lovable! Han Solo is a rogue. Everybody loves Han!
"Rogue" is a techbro euphemism for "we want you to think of this poorly programmed software as human so you'll engage with it more and more while we destroy your neighborhood with datacenters."