@dalias I know that's not the case because I also have access to such tools (made by others) and as I can run them on my own code I can see what they do and what they can find.
AI powered code analyzers is a real thing, and they make better code analyzers than the ones without the AI component.
@bagder Most of "AI" is human labor misrepresented as something the machine did, to defraud customers and investors.
I wouldn't be surprised if we find out that their "AI-powered code analyzer" did little or nothing here, and that they spent a lot of money on actual labor for the sake of promoting their product.
@dalias I know that's not the case because I also have access to such tools (made by others) and as I can run them on my own code I can see what they do and what they can find.
AI powered code analyzers is a real thing, and they make better code analyzers than the ones without the AI component.
@bagder It's certainly plausible that they find common patterns of error better than a simple grep would. But my default hypothesis will always be that this is marketing.
@bagder I do wonder though if you have access to the tools and the tools themselves can find the errors themselves, why it took Aisle employees reporting them in order for them to be found.
@dalias all these tools, like code analyzers have for ages, find and easily report a lot of things. The filtering, the assessing and the confirming still need a human involved to get really good.
@bagder Yes, so the majority of actual vuln-finding work is human labor and the AI tool is just giving you a huge list of mostly false positives...
@dalias you can easily also try that by running the AI powered code review tools. Clearly they work and find things. It's not a conspiracy.
Honestly, I do not understand what they mean...
@schnedan I don't claim to understand either, but I have worked with them for a while and I have received many reports generated from their toolset. They're good.
@bagder cudos to you for not becoming an "AI" hag. Although you had and still have every reason to.
After all its a tool like any other.
@bagder yeah, I asked one of these agents to scan for security holes a daemon I wrote with security in mind and it found like 5 DoSes I had not considered
@Migueldeicaza in many ways they are truly next-level and can find so many things we didn't see before. Code that previously had zero defects detected now suddenly has a lot of them...
@bagder @Migueldeicaza security code reviews are brutal to do, especially if the codebase is large
There’s almost no way an LLM won’t outperform a human doing this stuff
@joshbressers @Migueldeicaza I've been most impressed by them when they also "know" lots of details about the surrounding: like the protocols involved and 3rd party library's APIs etc, so they can detect when code abuses protocols specs or bad assumptions about data that is returned from another library etc. Those in-between areas that are so hard for humans to keep track of and see the mistakes in are perfect to have AI scan and poke holes at.
