Here's a comment about this from Stanislav Fort himself at Aisle: https://www.lesswrong.com/posts/7aJwgbMEiKq5egQbd/ai-found-12-of-12-openssl-zero-days-while-curl-cancelled-its
Discussion
Loading...
@bagder This is all so great and a plausible use of the technology, only if not fueled with hype and click-bait: saying that LLM is successful here instead of "AI" would sound more plausible to me. But likely less audience (CEOs) would buy that, so it's all gotta be "AI" hand-waving 😒 Surely enough it's far more advanced than fuzzy testing, pity there are little details on the actual technology published, just hype.
Here's a comment about this from Stanislav Fort himself at Aisle: https://www.lesswrong.com/posts/7aJwgbMEiKq5egQbd/ai-found-12-of-12-openssl-zero-days-while-curl-cancelled-its
@bagder besides the point but... grinding my teeth at calling these zero-day vulnerabilities, words have meanings, ffs, we aren't gonna call a vulnerabilities zero-day if they are following standard responsible disclosure, are we?
@bagder do you have any interesting contextually important information about what they mean when they saying ai reasoning system? E.g. are they describing a somewhat conventional and non-new deep learning static analysis as being an AI reasoning system product?
@screwlisp I don't have any insights into their engine and work flow. I have only seen the output from that, and that only after their human curation.
@bagder
Thanks. When they say
> have been building an automated AI system for deep cybersecurity discovery and remediation
it does not sound like they are talking about slopbots being infosec employees, and does sound quite a lot like deep learning code analysis.
@bagder Most of "AI" is human labor misrepresented as something the machine did, to defraud customers and investors.
I wouldn't be surprised if we find out that their "AI-powered code analyzer" did little or nothing here, and that they spent a lot of money on actual labor for the sake of promoting their product.
@dalias I know that's not the case because I also have access to such tools (made by others) and as I can run them on my own code I can see what they do and what they can find.
AI powered code analyzers is a real thing, and they make better code analyzers than the ones without the AI component.
@bagder It's certainly plausible that they find common patterns of error better than a simple grep would. But my default hypothesis will always be that this is marketing.
@bagder I do wonder though if you have access to the tools and the tools themselves can find the errors themselves, why it took Aisle employees reporting them in order for them to be found.
@dalias all these tools, like code analyzers have for ages, find and easily report a lot of things. The filtering, the assessing and the confirming still need a human involved to get really good.
@bagder Yes, so the majority of actual vuln-finding work is human labor and the AI tool is just giving you a huge list of mostly false positives...
1+ more replies (not shown)
@dalias you can easily also try that by running the AI powered code review tools. Clearly they work and find things. It's not a conspiracy.
Honestly, I do not understand what they mean...
@schnedan I don't claim to understand either, but I have worked with them for a while and I have received many reports generated from their toolset. They're good.
@bagder cudos to you for not becoming an "AI" hag. Although you had and still have every reason to.
After all its a tool like any other.
@bagder yeah, I asked one of these agents to scan for security holes a daemon I wrote with security in mind and it found like 5 DoSes I had not considered
@Migueldeicaza in many ways they are truly next-level and can find so many things we didn't see before. Code that previously had zero defects detected now suddenly has a lot of them...
@bagder @Migueldeicaza security code reviews are brutal to do, especially if the codebase is large
There’s almost no way an LLM won’t outperform a human doing this stuff
@joshbressers @Migueldeicaza I've been most impressed by them when they also "know" lots of details about the surrounding: like the protocols involved and 3rd party library's APIs etc, so they can detect when code abuses protocols specs or bad assumptions about data that is returned from another library etc. Those in-between areas that are so hard for humans to keep track of and see the mistakes in are perfect to have AI scan and poke holes at.