There's an anecdote about curl, but I'm not sure if there's enough in here to draw conclusions from. 20% of vulnerabilities have been made public, and many of those are marked as "withdrawn" or "duplicates". However, it's unclear whether the remaining 80% have not been made public because they're not fixed or real. In other words, this is analysis of survivors (public); not much can be said about those still under embargo.
I'd expect the remaining 80% are delayed not because there's something statistically special about them, but from the massive workload thrown at humans in the loop.
Anthropic needs to teach their systems to propose a fix too. Detecting a coding problem and proposing a patch is what we should expect from a company with supposedly the best coding LLM.
> Anthropic needs to teach their systems to propose a fix too. Detecting a coding problem and proposing a patch is what we should expect from a company with supposedly the best coding LLM.
Of course it's trivial to have an LLM generate a fix, too. They probably do. It doesn't mean a project (1) wants to take an LLM-generated fix at all, or (2) can immediately commit an LLM-generated fix without review (which takes time).
Claude is perfectly capable of fixing most vulnerabilities; if that's what maintainers want, they can just ask for it. If the concern is cost, Anthropic offers a Claude for OSS program.
The reason Anthropic isn't proactively sending patches is, I suspect, that those patches would not be welcomed, because trust in the models isn't currently at the point where maintainers are willing to merge them without human review, which takes approximately as much effort as writing the patches themselves. You can imagine the wailing and gnashing of teeth about "slop PRs" and "extractive contributions" that'd be all over HN.
And on top of that, it's a bit negative. How many millions of dollars is one perfect-10 scoring CVSS finding, total destruction style RCE, found in curl worth? Considering how many people use it.
It's worth zero millions dollars to curl. It might be worth something to attackers, but "wide use" is not the same as "the ability to compromise everyone" because most of the time, curl isn't used to automatically fetch untrusted URLs in a way that's controlled by the attacker from an environment they care about. It's worth much, much less than a 0-day in an iPhone or PHP. You'd need a buyer with a very specific itch to scratch and there won't be a bidding war.
By extension, if a vuln is worth something to attackers, it's probably also worth some fraction of it to their potential victims. But if you're an interesting target, your defense can't be based on the ability to find out about every single software defect ahead of the time. You need layers. And if you have that, the importance of proactively acquiring every 0-day is diminished.
A client-side bug in a command line utility that is mostly used as a debugging tool or glue in shell scripts where the URL is not attacker controlled? Probably close to nothing. Maybe if you can find some other software which lets you run curl with an attacker controlled URL, you could package it up as an exploit for that piece of software, but it's not a common config.
The most likely explanation is that it isn't worth submitting a massive stack of bullshit to human review.
If they actually were sitting on a PR gold mine of vulnerabilities they wouldn't be throttled on review resources while artificially inflating/misreporting their numbers. They would be diverting resources to clear the review queue.
It is bad if this points to a general pattern of AI being better at finding than fixing vulnerabilities. The larger that ratio, the harder to secure all the things. The current stat from this experiment is 37,966 to 217. Lots of false positives, but not enough to change the big picture. I hope they didn't apply equivalent tokens to both sides, or AI may be better at demolition than construction.
If anyone here hasn't experienced agents going into a loop finding "problems" and "fixing" them by adding more brittle complexity which introduces 10 more problems I'd be surprised. It's like the biggest experience of working with agents RN
The worst part is being thorough takes so much more work - they often do raise valid problems but you just can't set them in a loop because of this tendancy so you end up with way more work and hard questions
I have a "review panel" skill, that will do multiple rounds. But after two rounds, it looks for hot spots. If hot spots are found, it triggers a "big picture" design review.
yeah, thats similar to our current approach. It was inspired by a blog post from anthropic where they talked about the same kinda idea, where iirc after 3 rounds the author is asked to rearchitect.
The only other bit we also stole from that was the idea of only looking for issues in the original diff rather than spawning new problems from each diff fixing the previous. Of course the possibility exists of bugs being introduced while fixing others but we try to handle that with human review for the minute because its not worth getting agents to cover that bit, at least with naive approaches
have you tried modularizing / breaking down your code into different files (ideally each having functional code so they don't depend much on anything else) as much as possible?
are you suggesting this because you think I haven't tried it? I'm genuinely confused. We are professional developers working with a team of more than 1 person. Yes we have code in multiple files and have heard of the basic ideas of software architecture.
It's like being asked if i'm using source control? It's kinda a given if you're doing more than hobby projects.
In my opinion, what tells a lot about AI provider AGI claims versus reality is the difficulty to have LLM provide the fix to issues.
For PHP2shell vulnerability for example, there was "patched" .deb (and equivalent) packages only for the most recent and most trendy distributions but countless older versions of distributions and php versions were like abandoned without simple solution.
I would have guess that "patches" and pre-built fixed packages would be flooding the world. And with just a prompt of one line, most servers would be properly patched. But no.
Very interesting that the conclusion of the article is so tightly aligned with the VulnCheck's business model! It's almost as if this is just a thinly veiled advertisement...
> The CVEs are a sliver of this year's total, almost none are being exploited, fixes have stalled while duplicates and withdrawals pile up, and the triage burden is shifting onto the open source maintainers Glasswing promised to help. The receipts keep coming in. They still don't reconcile.
> VulnCheck is helping organizations not just to solve the vulnerability prioritization challenge - we’re working to help equip any product manager, CSIRT/PSIRT or SecOps team and Threat Hunting team to get faster and more accurate with infinite efficiency using VulnCheck solutions.
Given how fast progress has been in the last year, be careful what you wish for...
For example, as others have noted, there were no results on cryptography in OpenAI's October 6th math drop. But it seems unlikely that they didn't look at this sort of thing.
But PHP thing was a lot of drama over something that was not super large easy to exploit hole. Just a normal "have to patch it and bye" medium kind issue.
People say this vulnerability was "not super large easy to exploit" because of one poorly-informed skeptical blog post that hit the top of HN, but the evidence strongly suggests that post was simply wrong, and we're just going to have to live with the misinformation until it decays.
At this point the play is simply to move the goalposts, suggesting that vulnerabilities don't matter so long as you yourself couldn't easily exploit them, regardless of what well-armed attackers can do with them. That's, obviously, not how software security works.
You can have a lot of vulnerabilities without necessarily having working exploits to them, so it matters if it's just a lot of vulnerabilities without a lot of exploits.
Does the author expect that every company using Glasswing/AI is going to credit the AI directly? I would not expect Microsoft or Cisco to credit Anthropic if one of their internal teams uses AI to discover a CVE in their code.
There's an anecdote about curl, but I'm not sure if there's enough in here to draw conclusions from. 20% of vulnerabilities have been made public, and many of those are marked as "withdrawn" or "duplicates". However, it's unclear whether the remaining 80% have not been made public because they're not fixed or real. In other words, this is analysis of survivors (public); not much can be said about those still under embargo.
I'd expect the remaining 80% are delayed not because there's something statistically special about them, but from the massive workload thrown at humans in the loop.
Anthropic needs to teach their systems to propose a fix too. Detecting a coding problem and proposing a patch is what we should expect from a company with supposedly the best coding LLM.
> Anthropic needs to teach their systems to propose a fix too. Detecting a coding problem and proposing a patch is what we should expect from a company with supposedly the best coding LLM.
Of course it's trivial to have an LLM generate a fix, too. They probably do. It doesn't mean a project (1) wants to take an LLM-generated fix at all, or (2) can immediately commit an LLM-generated fix without review (which takes time).
Claude is perfectly capable of fixing most vulnerabilities; if that's what maintainers want, they can just ask for it. If the concern is cost, Anthropic offers a Claude for OSS program.
The reason Anthropic isn't proactively sending patches is, I suspect, that those patches would not be welcomed, because trust in the models isn't currently at the point where maintainers are willing to merge them without human review, which takes approximately as much effort as writing the patches themselves. You can imagine the wailing and gnashing of teeth about "slop PRs" and "extractive contributions" that'd be all over HN.
And on top of that, it's a bit negative. How many millions of dollars is one perfect-10 scoring CVSS finding, total destruction style RCE, found in curl worth? Considering how many people use it.
It's worth zero millions dollars to curl. It might be worth something to attackers, but "wide use" is not the same as "the ability to compromise everyone" because most of the time, curl isn't used to automatically fetch untrusted URLs in a way that's controlled by the attacker from an environment they care about. It's worth much, much less than a 0-day in an iPhone or PHP. You'd need a buyer with a very specific itch to scratch and there won't be a bidding war.
By extension, if a vuln is worth something to attackers, it's probably also worth some fraction of it to their potential victims. But if you're an interesting target, your defense can't be based on the ability to find out about every single software defect ahead of the time. You need layers. And if you have that, the importance of proactively acquiring every 0-day is diminished.
A client-side bug in a command line utility that is mostly used as a debugging tool or glue in shell scripts where the URL is not attacker controlled? Probably close to nothing. Maybe if you can find some other software which lets you run curl with an attacker controlled URL, you could package it up as an exploit for that piece of software, but it's not a common config.
Curl is also a library. Just saying :-).
The most likely explanation is that it isn't worth submitting a massive stack of bullshit to human review.
If they actually were sitting on a PR gold mine of vulnerabilities they wouldn't be throttled on review resources while artificially inflating/misreporting their numbers. They would be diverting resources to clear the review queue.
It's fake and it always was fake.
It is bad if this points to a general pattern of AI being better at finding than fixing vulnerabilities. The larger that ratio, the harder to secure all the things. The current stat from this experiment is 37,966 to 217. Lots of false positives, but not enough to change the big picture. I hope they didn't apply equivalent tokens to both sides, or AI may be better at demolition than construction.
So far the trend is pointing to AI being better at finding problems than designing software to avoid them.
Which I'm fine with really, but it means you need to be very careful letting it iterate on code until it finds no problems.
If anyone here hasn't experienced agents going into a loop finding "problems" and "fixing" them by adding more brittle complexity which introduces 10 more problems I'd be surprised. It's like the biggest experience of working with agents RN
The worst part is being thorough takes so much more work - they often do raise valid problems but you just can't set them in a loop because of this tendancy so you end up with way more work and hard questions
I have a "review panel" skill, that will do multiple rounds. But after two rounds, it looks for hot spots. If hot spots are found, it triggers a "big picture" design review.
It's been helpful.
yeah, thats similar to our current approach. It was inspired by a blog post from anthropic where they talked about the same kinda idea, where iirc after 3 rounds the author is asked to rearchitect.
The only other bit we also stole from that was the idea of only looking for issues in the original diff rather than spawning new problems from each diff fixing the previous. Of course the possibility exists of bugs being introduced while fixing others but we try to handle that with human review for the minute because its not worth getting agents to cover that bit, at least with naive approaches
have you tried modularizing / breaking down your code into different files (ideally each having functional code so they don't depend much on anything else) as much as possible?
are you suggesting this because you think I haven't tried it? I'm genuinely confused. We are professional developers working with a team of more than 1 person. Yes we have code in multiple files and have heard of the basic ideas of software architecture.
It's like being asked if i'm using source control? It's kinda a given if you're doing more than hobby projects.
Coding is hard. I wouldn't be surprised if any patch they suggest came with new bugs, as it often happens with humans.
It'll never scew to far of as long as defence is a RL target. Better red team just means the blue team has stronger target environmenta to RL for.
> Of Anthropic’s 37,966 reported findings, only 7,349 (19.4%) have been made public and just 217 have been marked fixed.
So the article is making a sweeping statement based on the 19.4% of publicly disclosed CVEs?
That statistic suggests the opposite. I'm sure government is aware of many back doors they keep secret and prefer them to remain secret.
"public" here has nothing to do with government holding on to vulns. it's about landing fixes and coordinating complicated / impactful findings
Expected this to happen maintainers overloaded with issues majority of which are nothing burgers.
In my opinion, what tells a lot about AI provider AGI claims versus reality is the difficulty to have LLM provide the fix to issues.
For PHP2shell vulnerability for example, there was "patched" .deb (and equivalent) packages only for the most recent and most trendy distributions but countless older versions of distributions and php versions were like abandoned without simple solution. I would have guess that "patches" and pre-built fixed packages would be flooding the world. And with just a prompt of one line, most servers would be properly patched. But no.
Very interesting that the conclusion of the article is so tightly aligned with the VulnCheck's business model! It's almost as if this is just a thinly veiled advertisement...
> The CVEs are a sliver of this year's total, almost none are being exploited, fixes have stalled while duplicates and withdrawals pile up, and the triage burden is shifting onto the open source maintainers Glasswing promised to help. The receipts keep coming in. They still don't reconcile.
> VulnCheck is helping organizations not just to solve the vulnerability prioritization challenge - we’re working to help equip any product manager, CSIRT/PSIRT or SecOps team and Threat Hunting team to get faster and more accurate with infinite efficiency using VulnCheck solutions.
I think the most obvious chicken little sign has been that we've had no catastrophic exploits in the past year due these findings.
Sure LLMs have helped reduce the attack surface and we patch things faster too, but I was really expected a few more Supermarket-level bangers.
Given how fast progress has been in the last year, be careful what you wish for...
For example, as others have noted, there were no results on cryptography in OpenAI's October 6th math drop. But it seems unlikely that they didn't look at this sort of thing.
There was that PHP thing, as well as *sshole, and how do we know the supermarket attackers didn't use AI?
But PHP thing was a lot of drama over something that was not super large easy to exploit hole. Just a normal "have to patch it and bye" medium kind issue.
People say this vulnerability was "not super large easy to exploit" because of one poorly-informed skeptical blog post that hit the top of HN, but the evidence strongly suggests that post was simply wrong, and we're just going to have to live with the misinformation until it decays.
At this point the play is simply to move the goalposts, suggesting that vulnerabilities don't matter so long as you yourself couldn't easily exploit them, regardless of what well-armed attackers can do with them. That's, obviously, not how software security works.
But imagine the potential fallout if so many services didn’t get so scared that they preemptively took themselves offline.
Apply sarcasm or not based on sleep deprivation.
We've had catastrophic vulnerabilities revealed every other day for months, what are you talking about?
Vulnerabilities and exploits are different things.
You mean like Notion being breached, refrigerators being turned off, that kind of thing?
Those are exploits. They are different from vulnerabilities.
Yeah, I know, and we've had a deluge of both lately. I'm not sure what constructive point you're making.
You can have a lot of vulnerabilities without necessarily having working exploits to them, so it matters if it's just a lot of vulnerabilities without a lot of exploits.
AI models generating slop findings? Who ever could have predicted this?!
Does the author expect that every company using Glasswing/AI is going to credit the AI directly? I would not expect Microsoft or Cisco to credit Anthropic if one of their internal teams uses AI to discover a CVE in their code.
Attribution/reporting is probably part of the conditions for these companies to get access.