Post by @0xabad1dea@infosec.exchange

Project Glasswing:

Claiming to have found 26 thousand real vulnerabilities but only 0.8% of them have resulted in a real fix in a real project after five months is dire. They blame it on the human independent review bottleneck, but human experts being paid for their time definitely have a higher throughput than that when working with data that’s actually actionable.

The assigned-at-Claude severity ratings are also dire. It assigns “high” or “critical” to 91% of findings. Most findings in the real world are low or medium. This should be especially true when using a magic machine to shake out every last little issue that was overlooked by humans focused on the biggest risks.

Together this implies it’s generating thousands of trivial or nonsensical findings and labeling them HIGH DANGER CRITICAL MUST FIX, and the human independent verifiers are sifting for the rare needle in this haystack worth passing on. This isn’t really an improvement over the high-noise automated scanners we already had

(This is a corporate blog of someone with their own vulnerability management services to sell, so apply an appropriate number of grains of salt to their analysis. Filter keywords: AI LLM Anthropic)

https://www.vulncheck.com/blog/anthropic-glasswing-receipts

VulnCheck

Vulnerability intelligence that predicts avenues of attack with speed and accuracy.

6 Boosts

0 Likes

The Liquid Schwartz

@OvertonDoors@infosec.exchange 3 hr. ago

@0xabad1dea

Anthropic needs another pallet of cash to keep the gas turbines spinning, so, WOLF BIG BADY BAD BAD WOLF!!!1!1Q

It's a VC cash plea posing as an actionable threat assessment. Anthropic's core talent is turning capital into waste heat.

David Chisnall (*Now with 50% more sarcasm!*)

@david_chisnall@infosec.exchange 6 hr. ago

@0xabad1dea Just before they did the ‘security’ stuff, someone sent me a ‘comprehensive code review’ of one of my projects. Most of the performance things were lifted directly from open issues I’d written, but one that it flagged as high importance was an intentional design decision.

It pointed out that one operation does a broadcast cache invalidation that invalidates all caches, rather than the subset that it actually needs to. The recommendation was to switch to having fine-grained cache state tracking. The problem is, that actually made cache accesses more expensive on the most performance-critical path in the system. Cache invalidations happened less than once every million accesses in typical programs, so this was an incredibly unusual case and optimising for it hurt performance everywhere else.

But, because it didn’t actually understand the code or the context, it saw a pattern that was slow and matched it to a high-severity warning.

I would expect it to do exactly the same thing with security reports. Memory-safety bug in a code path that can be reached only via trusted data? Highest severity!

Dr. Bruce Simpson

@bms@mastodon.bsd.cafe 2 hr. ago

@david_chisnall @0xabad1dea Revocation of capability or RBAC access control *behind* the capability grant does present an exception to usual received wisdom for soft state vs hard state, recalling Noel Chiappa's original note on the subject; it's closer to his definition of "critical state".