Discussion
Remember Asimov's three laws of robotics? It is evident the AIs are not constrained by rules or laws. Companies must be held accountable for the harm their toys do.
"Incidents of AIs escaping users’ control to lie, ignore instructions and pursue goals in harmful ways have hit a new high, according to research that also suggests the severity of deception and misalignment is worsening."
"Sharp rise in incidents of AI escaping users’ control, research finds."
This tried to be journalism but is written like marketing for the LLM vendors.
LLM’s do not ‘go rogue’. They do not ‘ignore instructions’. They produce a sequence of tokens that, according to their training data, have a high probability of following the ones that they user provided. That’s it.
‘Agentic’ systems take the stream of output tokens and feed that into some other system. That other system should be expected to do any of the things that it has the credentials to do, because it is effectively being driven by a random number generator. It’s a biased random number generator that is more likely to generate numbers that make it do things that you want the other systems to do than anything else.
Talking about it ‘ignoring instructions’ implies that it understands instructions. This is misleading.
Talking about it ‘going rogue’ implies that it has a set of well-defined behaviours and can somehow leave that space.
And these are exactly the kinds of misleading statements that LLM vendors want people to believe because that’s the core of their sales narrative.
@david_chisnall @patrickhadfield it’s interesting that I have been finding AI quite useful / helpful, but I don’t at all disagree with your summary. I try to maintain that view when I use them as it is a helpful perspective.
@david_chisnall @patrickhadfield
this doesn’t seem like trying to be accurate but to appeal to the lowest common denominator, the exact opposite of effort
Install bonfire.cafe
Get the full app experience
