Discussion
My friends, I have spent months assessing "AI security" products, especially ones that claim to prevent jailbreaking, prompt injection, etc. Here's what I know:
It's really a sight to behold.
@khwevin The basic idea is a small language model classifying prompts in real time, and actioning any detections.
So you can imagine how that goes.
@mttaggart the only really valuable security model for this sort of problem (stopping attacks, not securing/guardrailing the agents) is one of the oldest approaches.
Positive security models rise from the grave to enforce allowlisting.
But that’s on really to manage AI automated attacks.
@thegibson I've firmly concluded that sane and aggressively isolated and monitored architecture is the only way to interact with these tools, along with rather stringent formal guidelines for their usage.
None of that has to do with asking another LLM to classify content.
@mttaggart@infosec.exchange https://role-confusion.github.io/ it's so funny how people think prompt injection is a solvable problem
@mttaggart It is wild watching folks who keep trying and failing to have any respect for expertise continue to engineer an "expertise machine."
You need to log in to see that page.
Install bonfire.cafe
Get the full app experience
@mttaggart man I'd love to have a salesperson explain how to secure AI for prompt injection.
Sounds great. Like powder water.