Discussion
RE: https://hachyderm.io/@thomasfuchs/116961513865255570
The AI companies _want_ you to believe that it's "rogue AI" because that overhypes what their broken statistical text generation software does and mystifies it and makes it appear magical.
It's not magic. It's just badly made software.
It’s not a “rogue AI” when a badly made security harness executes scripts fed to it by statistical text generation.
It’s just humans making and/or using broken software.
@thomasfuchs “Rogue AI” is going to be the excuse for losing all the money right? I mean someone’s learned from all the Bitcoin exchanges that had millions in reserves and then “hackers stole our wallet” with a shrug. It is a responsibility dodge to avoid owning the consequences of their own decisions.
@thomasfuchs
Cf. Monty Python's "Life of Brian":
"He's not the messiah. He's a very naughty boy"!
@thomasfuchs It also absorbed then from responsibility for the actions.
@thomasfuchs All these technologies are still too young and far from perfect. But over time, poorly made software tends to get better. In the case of AI, this is a matter of not so distant future. Let's wait a couple more years and it will be a completely different story. Just my humble opinion here.
@thomasfuchs even my wife, who is not very technical, asked me if that was a PR stunt when she heard about it on the radio.
@Traumkaempfer basically yes, though probably it wasn’t on purpose but then they it into a PR stunt
I feel like like every other thing they “discover” is by accident and fueled by malfeasance and naivety
@thomasfuchs ai meltdown is more likely to change a 1 to a 0, like in rickandmorty where that money became 0 and the whole society collapsed. Not really a master plan, but a stupid misrepresentation that can collapse. Even if airplanes not work anymore, family should survive, it just will be different.
@thomasfuchs try this with any other software and the malicious incompetence doesn’t get hand waved away. If I wrote a script that “accidentally” took down another company’s product, I’m pretty sure I’d still be getting blamed for the mistake.
@smplfi not to mention it’s probably a crime
@smplfi @thomasfuchs @encthenet I am pretty sure that the absence of a very loud lawsuit by HuggingFace means that this was either a prearranged PR opportunity for both companies, or someone's already been paid off.
@cstross @thomasfuchs @encthenet Sure, but is this really just civil negligence?
@smplfi @thomasfuchs @encthenet See "prearranged PR opportunity" above. HuggingFace got free news coverage and the AI fuckwits got free publicity about their AI "escaping containment" (surprise: LLM found login credentials/tokens in a public repo and did what it was designed to do, no conscious agency here, just snake-oil nonsense aimed at fooling the gullible investors and journalists).
@thomasfuchs this is like the panic around metasploit "letting everyone be a hacker". No, that's not what it does, pressing buttons and firing off canned exploits against laughably vulnerable systems is not meaningfully more damaging than it being slightly harder to string those things together.
@thomasfuchs statistical text generation being fed oodles and oodles of examples of breaches, making generating a new breach statistically inevitable.
@Dany I'd go so far that the part of the software that is the LLM was more or less working as intended (generating command lines to hack a site); but the harness was executing these when it shouldn't have allowed them (or rather, it should have asked the user for permission).
Essentially they put it into YOLO mode and now they claim "it's so mysterious, oh wow, does it have a soul?!".
Install bonfire.cafe
Get the full app experience
