Anthropic says Claude Mythos is too dangerous to release. Is vuln research basically cooked now?
Posted by choochilla44@reddit | ExperiencedDevs | View on Reddit | 15 comments
Read here: https://shiftmag.dev/claude-mythos-opens-the-cybersecurity-pandoras-box-9622/ that Anthropic just announced Claude Mythos, a new internal model they claim is too powerful to release publicly and according to them, it’s already finding real vulnerabilities across major OSes, browsers, widely used libraries etc.
Access is ofc limited to a small group of orgs, supposedly to avoid handing bad actors a loaded weapon.
At first I rolled my eyes a bit at the 'too dangerous to release' thing, we’ve definitely seen that movie before...
but i just i wanna know if you think this is actually a big shift or just the same thing with better tools?
throwaway_0x90@reddit
It may or may not be true, but this whole thing around Mythos is just hype nonsense so you should ignore it until something
very concretehappens.ExperiencedDevs-ModTeam@reddit
Rule 9: No Low Effort Posts, Excessive Venting, or Bragging.
Using this subreddit to crowd source answers to something that isn't really contributing to the spirit of this subreddit is forbidden at moderator's discretion. This includes posts that are mostly focused around venting or bragging; both of these types of posts are difficult to moderate and don't contribute much to the subreddit.
HwanZike@reddit
Ask yourself what is the point of announcing it if it's not something you can/will release. Instead of either just not saying anything in public or just releasing it.
liquidbreakfast@reddit
i mean, mythos was announced over a month ago so i feel like this has been discussed to death, but...
if you've ever used claude to write anything, you've probably noticed its tendency to include a lot of null guards, input sanitization, etc. in extremely unlikely places, ie where it's a closed system with full control of inputs, or where a user would already need root access to do the thing it's protecting. a human developer would probably be too lazy to add those, or find the cost/benefit to not really make sense.
the reality is that almost all code is riddled with low severity, low exploitability issues and imo the difference with mythos is mostly just that it doesn't do that equation - it can just belt and suspenders everything and call it finding vulnerabilities.
liquidbreakfast@reddit
i mean, mythos was announced months ago so i feel like this has been discussed to death, but...
if you've ever used claude to write anything, you've probably noticed its tendency to include a lot of null guards, input sanitization, etc. in extremely unlikely places, ie where it's a closed system with full control of inputs, or where a user would already need root access to do the thing it's protecting. a human developer would probably be too lazy to add those, or find the cost/benefit to not really make sense.
the reality is that almost all code is riddled with low severity, low exploitability bugs and imo the difference with mythos is mostly just that it doesn't do that equation - it can just belt and suspenders everything and call it finding vulnerabilities.
EnderMB@reddit
I'd have believed them if they hadn't been pumping this shite out for years now, saying the same thing every time...
We're quickly relaxing the point where a lot of people are wondering if the AI costs are worth it. When you're spending thousands on tokens for little productivity benefit, and all you hear each year is "but our next release will steal everyone's jobs, honest!" the truth is that we've reached a point where LLM's have peaked.
funbike@reddit
I'm all for this, for now. If executed well, this could make the world a safer place.
License it to orgs. Partner with github, and similar services, to offer it for any user/org that wants to scan their project. Only those with commit access can use it and see results. Make it free for small open source projects, and extremely cheap for huge open source projects.
Then, after most projects have been scanned, release it as a cloud vuln scanning service for source code, but only allow for authors/owners of the code to submit. Still don't provide direct LLM access as something you can prompt to go attack a web site.
Okay, then release it only to white hat security researchers, whom promise 90 day CVD (coordinated vuln disclosure). By this time most projects should be fairly safe.
After enough time and industry-wide hardening, they could finally provide open access to the LLM's API. Perhaps after fine-tuning to discourage malicious intent.
ClideLennon@reddit
- Sam Altman
ClideLennon@reddit
No, they are just lying for hype.
https://www.youtube.com/watch?v=-j6dQK_MggU
RelevantJackWhite@reddit
It's literally in the name. Mythos...
TheTacoInquisition@reddit
"We'd love to release a new product, but it's just WAAAAAY too good at it's job, honestly, trust us!".
It's bluster to pump up investors. If it's dangerous, it's more likely it finds issues in code anthropic models create, and they don't want bad press.
notoriou5_hig@reddit
They scanned with it at my work and it has a pretty high false positive rate. Your mileage may vary, but it’s been a thorn in my side for a bit now just reviewing AI-generated tickets and deciding if they’re real or not.
pydry@reddit
It's a ploy to keep that bubble pumped. The financials of anthropic are fucking cooked and they are trying to delay the trough of disillusionment from kicking in before their IPO.
VanillaCandid3466@reddit
It's got nothing to do with their upcoming IPO ... honestly, I swear it ... nada ... nothing ... 😃
shinto29@reddit
It's surely a stronger model than the latest Opus but it's purely marketing. And it's working, I don't think there's much to be worried outside of what bad actors already have available.