Why do LLMs fall for prompt injection attacks that wouldn’t fool a fast-food worker?

In this piece, Fastly Distinguished Engineer Barath Raghavan and security expert Bruce Schneier explain how AI flattens context—and why that makes autonomous AI agents especially risky.

A sharp, practical take on AI security. 🍔🤖: https://spectrum.ieee.org/prompt-injection-attack

#AISecurity #PromptInjection #LLMs #Cybersecurity

Why do LLMs fall for prompt injection attacks that wouldn’t fool a fast-food worker?

In this piece, Fastly Distinguished Engineer Barath Raghavan and security expert Bruce Schneier explain how AI flattens context—and why that makes autonomous AI agents especially risky.

A sharp, practical take on AI security. 🍔🤖: https://spectrum.ieee.org/prompt-injection-attack

#AISecurity #PromptInjection #LLMs #Cybersecurity

📜 𝗣𝗮𝗽𝗲𝗿 → https://arxiv.org/pdf/2501.01872
🌐 𝗣𝗿𝗼𝗷𝗲𝗰𝘁 → https://ukplab.github.io/emnlp2025-poate-attack/
💾 𝗖𝗼𝗱𝗲 + 𝗱𝗮𝘁𝗮 → https://github.com/UKPLab/emnlp2025-poate-attack

And consider following the authors Rachneet Sachdeva‬, Rima Hazra, and Iryna Gurevych (UKP Lab/TU Darmstadt) if you are interested in more information or an exchange of ideas.

(3/3)

#NLProc #LLMSafety #AIsecurity #Jailbreak #LLM

#oneflip : one flip to rule them all.

The linked paper by students of George Mason University (Xiang Li et al (2025): "Rowhammer-Based Trojan Injection:
One Bit Flip Is Sufficient for Backdooring DNNs") descibes how flipping a single bit suffices to corrupt the output of high-precision a.i.-s based on deep neural networks.

Of course there are no mitigations: none of the creators imagined malice.

#cybersecurity #aisecurity