“Any AI that is working in an adversarial environment—and by this I mean that it may encounter untrusted training data or input—is vulnerable to prompt injection. It's an existential problem that, near as I can tell, most people developing these technologies are just pretending isn't there.”
— Bruce Schneier
quoted in
@RuthMalan 🧵 #Agentic #AI #Security
“The fundamental security weakness of LLMs is that there is no rigorous way to separate instructions from data... This leads to the “Lethal Trifecta”: sensitive data, untrusted content, and external communication - the risk that the LLM will read hidden instructions that leak sensitive data to attackers. We need to take explicit steps to mitigate this risk by minimizing access to each of these three elements.” — Martin Fowler
