Discussion
I understand that Anthropic thinks they're doing open source projects a favor here, but what they're actually doing is creating an enormous amount of work for people that they are not paying.
https://www.anthropic.com/research/launching-opt-in-vuln-finding-service-for-open-source
Can you help me understand your complaint.
I wrote software for 40 years and I use Claude daily now. It is an amazing tool.
I think it really is a service to know where vulnerabilities are, for free. They can choose whether to do anything about it or not.
But I think of it like a medical diagnosis. You don't have to do anything about that tumor they found, but it may be worse if you don't.
@evacide original capitalism: find ways to externalize work and concentrate profits. AI derivative: find ways to monetize the process of externalizing the work
@evacide ... to avoid expensive litigation.
@evacide I think it is intentional sabotage while they get more data to fine tune their models.
@evacide I doubt that they consider it a favor. I strongly suspect that they consider it somewhere between a protection racket and releasing a disease so that they can sell the cure...
@evacide
This is perfect. They are getting open source maintainers to proofread their submissions and use this for training.
For free!
@evacide i cant even stand githubs dependabot i dont mind banning another
@evacide TBF we were getting those emails anyway and the Glasswing reports are both higher quality and come with .patch files as starting off points for fixes. The reports even triage the issues rather than report everything as a level 9 CVE.
As far as damage limitation goes then, it takes the edge of all those people with claude max subs who want CVE fame and for whom claude has replaced metasploit cli as the only security tool they know
@stevel Ah, now this is some actually useful feedback. If the maintainers themselves consider this to be an improvement on the status quo, that counts for a lot.
@evacide it's an improvement on the flood of "I'm a security researcher and I found a CVSS score 9.0 RCE CVE..." which comes down to an attack vector where stage 5 is "submit job as the authenticated user". There's a reason our SECURITY.md calls that out along with "if you own the local filesystem and configuration it doesn't count as an exploit" -and it is not mythos supplied reports.
https://github.com/apache/hadoop/security#before-filing-a-report-including-ai-assisted-reports
@evacide one package i use has gone from a few PR's a week to 10-50 a day I think.
@evacide The alternative if to agree and find them first or wait until open-weight models catch up and your vulnerabilities become common knowledge.
I really hope it's a temporary state of affairs and in the end the software can converge on a state when LLMs find no vulnerabilities - although intuitively that should be a lot of work.
@Oytis Unfortunately, I don't think that this where we are headed and even if we get there, both patching and exploitation are non-trivially hard problems.
@evacide it's opt-in tho, you don't have to if you don't want to
@evacide The projects that agree will most likely be the one already being paid to allow slop submissions so it'll basically be Claude checking it's own work. No way that could go wrong!
@evacide An excellent Illustration of the meaning of the German word “verschlimmbessern”.
Install bonfire.cafe
Get the full app experience
@evacide I like people who can admit when they’re wrong.