RE: https://mastodon.social/@hynek/116476031032569096
this is what I mean when I say genAI has got people deciding to act stupid on purpose. things like "prompt injection" are just things we previously recognised as glaring categories errors, but suddenly we can't recognise very obviously terrible ideas because they're wrapped up in the bullshit machine
“begging stochastic systems to be good boys is not a security strategy” exhibit ♾️
What really grinds my gears is how a whole industry chose to forget the 101 of ops security. This is not an “AI" problem, this is gross negligence by cloud providers like Raleway pretending "having evals" is an acceptable defense against catastrophic data loss & users having production tokens of any kind lying around such that an attacker (or misguided agent) can find it.
