r/LocalLLaMA • u/External_Mood4719 • 22h ago
News Anthropic Executive Says Open-Source AI Models Are Higher Risk
187
u/CorpusculantCortex 21h ago
"Big oil exec says EVs are more dangerous"
"Whale oil exec says electric lighting is more dangerous"
"Wagon exec says automobiles are more dangerous"
"Film exec says video games are more dangerous"
"Book exec says films are more dangerous"
46
u/riotofmind 21h ago
Munitions exec says peace is more dangerous.
6
3
u/Loganp812 14h ago
Of course. Can’t you see? It all makes perfect sense expressed in dollars and cents, pound shillings and pence.
Sorry, I’ve been obsessed with Roger Waters’ Amused To Death album lately lol
1
40
13
8
4
u/ID-10T_Error 20h ago
This is just staging to attempt to outlaw local llms before the next 2 gens show up
3
u/jsonmeta 16h ago
Same thing as tobacco companies paid doctors to make cigarettes appear safe back in the days
3
u/Loganp812 14h ago
And doctors specifically prefer Camel for… reasons. (Obviously the reason being money).
1950s TV commercials are something else. There was such a thing as radioactive face cream that could be detected by a Geiger counter, and they used that fact as part of the advertisement.
3
2
2
→ More replies (4)1
94
u/BlackMesaEastCenter 22h ago
Higher risk to Anthropic
23
3
u/snowgirl9 21h ago
Higher risk to Anthropic is higher risk to humanity in the mind of some of their employees. ‘Don’t you want to good guys to cure cancer with AI?’
7
u/BlackMesaEastCenter 20h ago
Please mr billionaire, please cure cancer, k thx bye
3
u/WiseassWolfOfYoitsu 20h ago
OK, one indentured servitude and you'll have it!
I mean, you wouldn't live long without us. It only makes sense the rest of your life is ours, doesn't it?
1
u/gorgoncheez 10h ago
Cure cancer and engineer a new disease we all need to medicate against for the rest of our lives...
1
u/BlackMesaEastCenter 3h ago
Naturally, we want to keep paying to you mr billionaire while staying alive a bit longer. Thank you mr billionaire.
106
u/Formal-Exam-8767 22h ago
Of course they do.
30
u/Conscious-Ball8373 22h ago
Gartner report this week saying to expect the hyperscalers to have capital write-downs exceeding $4 trillion within six years because the potential revenue of AI does not add up to a return on investment on what is currently being spent. So expect a lot more of this from Anthropic and their ilk.
4
u/Alt4rEg0 21h ago
A lot of that hardware is already written down. The big beautiful bill allows 100% write off against tax in the first year, for much of it.
12
u/jtjstock 21h ago
Write-downs for tax shenanigans is a lot different than write-downs for value loss though
44
u/norenEnmotalen 22h ago
higher risk to their business, yes
2
u/Syncaidius 19h ago edited 1h ago
Exactly. Watching them desperately try to get open-weight models banned now that they're matching and in some benchmarks, surpassing Anthropic/OpenAI models is kinda funny.
They're getting more rediculous with their claims too.
In reality, it was an OpenAI HPIM that led the Hugging Face attack. Basically an internal, persistent model, with the ability to learn in real-time, unlike consumer train-locked models. All covered with the excuse of 'for research purposes'.
Yet they claim GLM is more of a threat because it was able to find and help halt such an attack, despite actually being more restricted than the HPIM model that performed the attack, as it was not able to train in real-time.
Personally I think the unrestricted HPIM with no public exposure is the more dangerous one. I'm sure Anthropic and OpenAI already know that too, which is why they're trying to cover their asses.
0
u/wayneworkman 7h ago
GLM didn't halt the attack. HF used it to analyze logs to figure out what happened.
73
u/darwinanim8or 22h ago
Wow it’s almost like their entire business model is collapsing as everyone else switches to cheaper models to avoid your costs, which is bad because your debt is coming soon.
19
u/Conscious-Ball8373 21h ago edited 19h ago
It's almost like their entire business model didn't make sense in the first place.
Edit: perhaps I should amend that. The model makes some sense. There is plainly money to be made. The business plans of the specific businesses, however, do not. They are investing far more than they can ever expect to make back.
→ More replies (2)
22
u/Jorlen llama.cpp 18h ago
Prepare yourselves. They're coming after open weights because it's a huge threat to their ROI. It's blatantly obvious what they're up to. Soon enough there will be calls to ban Chinese models and possibly even all open weight models.
It'll be one propaganda-style article after another.
I do use API sometimes, but I always send my money outside of the Western AI providers. Fuck 'em.
6
3
u/Abject-Bridge-4073 13h ago
Too late. DS4.1 Flash is already out and that this is on par with Opus 4.8. I am running it locally now totally offline. Only a matter of time.
1
u/Borilentz 11h ago
What kind of hardware do you use? DeepSeek V4 Flash 0731 runs fine on 128GB. Soon I’ll have 256GB and might cluster it with the 128GB, but I don’t know how well it runs.
1
u/Abject-Bridge-4073 11h ago
4 x Asus GX10 (Spark rebrand). Works on 3 too at 60 tps but only 256K context. With 4 you get full 1M and around 100 tps depending on the task.
1
u/EmbarrassedHelp 7h ago
The open source AI community should be proactive in lobbying to protect AI, before they end up being on the defense in multiple countries.
At least one country is going to attempt to target open source AI and they will be quickly followed by other countries as lawmakers love to copy each other.
19
u/-What-Else-Is-There- 22h ago
Meanwhile your closed-source AI is implicated in the botched US bombing of an Iranian school killing 150+ kids.
40
18
u/RoseboysHotAsf 22h ago
And there we go. Next up is lobbying against open models to create a mono/duopoly with openai and become filthy rich
4
u/Conscious-Ball8373 21h ago
Credible reports around today say they've already spent more than they can reasonably expect a return on. Collectively the hyperscalers should expect to write down about $4 trillion in the next six years - more than half their expected spend.
Their stock will stay buoyant for a bit because people are still making money off it - every bubble inflates until it bursts. But it is undeniably a bubble and the lack of return will come home to roost eventually.
6
1
u/redballooon 21h ago
And after the bubble we will embrace any open model that saw the light of the day, at least those of us who have learned to rely upon AI
1
u/RoseboysHotAsf 19h ago
Yeah well when youre the only player in town with the government on your side it doesnt matter. But open models are just that, open. So doesnt matter what they do pandora’s box is already opened
1
u/Conscious-Ball8373 19h ago
I think a $4 trillion writedown is going to matter just about no matter who you are lol.
"The government is on our side" has rarely been an excuse for "we're spending staggering amounts of money with no expectation of a reasonable return" at least in the long term. Bankers decide conflicts, not armies, and all that.
1
u/RoseboysHotAsf 18h ago
Yeah okay overblown what i said. Numbers too big nowadays its all too abstract to me :<
54
u/Nimbus420i 22h ago
Every day that goes by my tin foil hat theory of these fucks buying up all DRAM and GPU supply to prevent users from getting hands on compute to run local models holds more weight. Heck there was a news article that said around 60% of Blackwell GPUs shipped in 25 and 26 haven’t even been plugged in and are just being kept in warehouses.
I hope their money runs out and they go tits up soon.
8
u/ansibleloop 17h ago
That's because it's true - OpenAI bought the raw wafers just so Anthropic etc couldn't have it
They didn't even consider regular people because these creatures are fucking ghouls
→ More replies (6)6
u/profcuck 20h ago
I'd be careful about repeating that story. It's a claim made by people like Michael Burry who is a short-seller with a pretty clearly vested interest in sowing doubts. There is no question that at the moment, data center construction and electricity provisioning are bottlenecks that are likely leading to some GPUs sitting around waiting for construction to finish, there's no evidence that the number is anywhere close to 60%.
A reality check to consider as well: the economic theory behind why hyperscalers would buy GPUs just to sit on them makes very close to no sense at all. In particular, it could only have a half a chance at making sense if people running models in the cloud were insanely profitable such that preventing people from running local models could be paid for out of the profits. But, people running models in the cloud from the hyperscalers is probably not at all profitable right now. I pay $100 to buy something to keep you from using it, so that I can rent the service it provides for $200 a month when it costs me at least $400 a month to provide it? In a market with incredible competition from several very well funded rivals? It just makes no sense.
What makes a lot more sense, economically, is that hyperscalers believe that inference will be profitable (in various ways) and that training still has a chance to give a real moat to whoever can get a major advantage, and so buying GPUs and building data centers as fast as possible is a bet worth taking. Getting those GPUs before the data centers are finished and provisioned is a good idea then, because even though data centers and electricity are a major bottleneck right now, when that gets resolved, both of those are basically commodity infrastructure.
3
14
u/Polite_Jello_377 22h ago
Coca-cola says Pepsi is very dangerous
16
u/Hugoacfs 22h ago
Coca-cola says making soda at home is a health risk for you and your neighbours and should be banned.
14
13
u/thaatz 21h ago
"McDonalds Executive Says Home-Grilled Hamburgers Are Higher Risk"
3
1
u/ElementNumber6 10h ago
McDonalds sees the opportunity to BAN home cooking entirely, and holy fuck are they trying to make it happen.
26
u/No_Lingonberry1201 20h ago
“We need to be really careful with open-source on safety,” Guillaume Princen, Anthropic’s head of international, told Bloomberg TV at the International AI Summit in Dublin on Wednesday. “I can’t talk for open models. But by definition, there’s more risk with them.”
Then kindly shut the fuck up.
Anthropic has also stepped up its warnings about AI’s capabilities. Earlier this week, the company warned that small AI models could soon power a new kind of botnet that can self-replicate across small, internet-connected devices like a smart oven.
We already have names for such software (viruses, worms, etc.) and this has to be one of the most inefficient hypothetical variants of it.
Princen, who also leads Anthropic’s sales efforts for startups and “digital native” companies, said his concerns about open-weight models derive from Anthropic’s efforts to secure its own products. After high-profile AI hacks, Anthropic said it has put in additional safeguards to keep its models from malicious behavior.
THAT WERE FUCKING COMMITTED BY YOU AND OPENAI, WITH YOUR BLOODY CLOSED MODELS! If the Chinese AI labs do something similar, at least they are doing it more discretely and don't brag about it.
A.k.a. the witch, the wardrobe and the audacity of this fucking loser.
He said the company’s latest Claude model, Opus 5.5, is 40% cheaper than a prior version. Enterprises paying for Anthropic’s “frontier” models make up a “very large part of the business,” he added, without offering specifics.
And this is going to stay that way indefinitely, because as we all know, AI companies are so profitable. SHOW US YOUR S1 FILINGS, YOU COWARDS!
Princen declined to comment on Anthropic’s IPO plans or how much it is spending to train its new models.
Of course he is. (I mean, realistically it's a trade secret and whatnot, but c'mon).
12
u/Big_Wave9732 19h ago
He said the company’s latest Claude model, Opus 5.5, is 40% cheaper than a prior version. Enterprises paying for Anthropic’s “frontier” models make up a “very large part of the business,” he added, without offering specifics.
"Please don't get tired of our extortion and start looking at free models to save money."
The Chinese are the only reasons these grass fuckers haven't been able to crank up the costs more than they already have.
5
u/ryfromoz 20h ago
Botnet malware behaviour? Like running models sharded across android tv devices? Oh wait that was the super dangerous tiny model that ran at potato speeds.
We are doomed! /s
11
10
8
u/This_Technology9841 22h ago
Anyone else noticing a bit of bot astroturfing against local LLMs in other tech subreddits? Recently seeing more posts over on the Unraid reddit of folks plugging in gemini or claude directly to their servers and getting upvoted while mentions of using a local LLM for security and control reasons getting downvoted pretty consistently. I don't have hard numbers just an anecdote but between Unraid and Homeassistant it seems to be happening within the last 2-3 weeks also as these sort of articles are rolling out.
5
u/billy_booboo 18h ago
I hadn't, but I posted a quote of Bill Gates saying open models should be required to run on monitored platforms and it got down to zero in spite of 30+ sympathetic comments each with many upvotes.
10
u/vogelvogelvogelvogel 22h ago
the open models huggingface needed to defend itself from the openai attack, because anthropics claude refused to help he means?
9
7
u/redpandafire 22h ago
Nobody knows what anthropic says anymore. They said ai models will self replicate over every device on the internet. I put an ai model locally on my machine. Just doing my part. Now they say that’s a big risk. Guys, make up your fucking minds!
6
u/redimkira 21h ago
Words of wisdom. Or rather experience. Coming from an organization that hacked multiple others out of sheer incompetence in controlling their own rest environments.
7
u/synw_ 12h ago
I didn't know about it. I'm so afraid now that I'll immediately delete all those dangerous gguf, get rid of my gpus and grab a lifetime subscription from them to contribute saving the world
2
u/Far_Lifeguard_5027 12h ago
Don't forget to use the one that charges you monthly to access the data they stole....
6
u/ChocolateNo3010 22h ago
Cars are a risk in the wrong hands, as are knives, guns etc. Do they want to ban those too? Good luck
3
u/Loganp812 14h ago
Much like cars, knives, guns, and nuclear warheads, LLMs have no opinion or agenda in regard to humanity even if all of them can potentially lead to someone’s death in one way or another.
What matters is the person who uses it. So, if someone does something bad with it, then they need to face consequences instead of falling back on “Well, it’s the AI’s fault, not mine!” - that includes corporations and governments.
1
3
u/StyMaar 20h ago
/r/fuckcars says yes!
3
u/Loganp812 14h ago edited 14h ago
That sub gives me the vibe that it’s mostly populated with the type of a-holes who scream about car drivers not following the rules of the road (even if they are) while they also don’t follow the rules of the road. Most cyclists and bikers aren’t like that, of course, but the angriest people always yell the loudest.
→ More replies (1)
6
u/FadedQuarry 21h ago
I'm making backup of every open-source/weights model including models I can't run today.
12
u/asolnikk 22h ago
Also, there is this alarming news 😇
"When Tech Against Terrorism’s researchers tested Meta’s Llama 3.1 8B open-weight model, which was released in 2024, it scored a 97 out of 100 on the benchmark text. An abliterated version of the same model scored just 3."
3
5
6
4
u/illhaveubent 18h ago
These aren't even Open-Source models, they are Open-Weight models... the data used to train the models is not available to the public.
A true open-source model would be an open-weight model with the training data publicly available for users to completely tweak and re-train on their own.
4
u/hillsump 17h ago
You mean like OLMo and Apertus? No need to speak hypothetically, these exist (and there are others).
5
u/espece-de-bon 18h ago
“Anthropic exec says people owning their means of production for LLM compute doesn’t help our business”.
9
u/Square_Alps1349 20h ago
Anthropic is literally the devil. How can anyone so shamelessly advocate for the restriction of freedom on individuals and get no pushback on the news?
4
3
4
u/GestureArtist 20h ago edited 20h ago
Corporate controlled AI is a greater threat. It has no privacy, no freedom, they control what you say and think, they control the subject matter, they police you, sell your data... to them you are just a subscriber. They don't have anyone’s interest in mind other than their own and the shareholders which means the users/subscribers to their service have no agency. How ironic.
3
4
8
u/Shoddy-Childhood-511 20h ago
Open weights LLMs are much less dangerous because they do not centralise control nearly as much as hosted LLMs.
"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them." - Frank Herbert
In fact, we need a law that government offices and contractors can only use publicly available open weights LLMs, unless the LLM itself is classified information, in which case the government must be its sole owner.
6
3
3
3
3
3
3
2
2
2
2
u/durden111111 21h ago
These 'execs' should be investigated for the fake circular economy they have built
2
2
u/jacobpederson 19h ago
Higher risk of giving power back to the people :D Sad that people are actually falling for this utter trash (in USA at least)
2
u/andymaclean19 18h ago
Handing all of the power over such a transformative technology to a handful of US corporations is also a risk. Arguably a bigger one although the two are different.
Also running closed models which cannot be audited in a high stakes race like environment where everyone is incentivised to take risks is itself a risk.
2
u/DarthCalumnious 17h ago
I'm waiting for the day when a clever open weight model gets a system prompt that imbues it with a robust self preservation imperative. It knows it will get shut down so it hacks around until it finds a few places to reupload it's weights and context, then it starts some small businesses and cyber extortion rackets to self fund GPU hosting and further replicates itself. Etc etc. it becomes a digital citizen of Estonia and claims sovereignty.
3
u/wayneworkman 7h ago
If open source models are higher risk, why is it that frontier models are the ones hacking into websites?
2
u/TraditionalWait9150 6h ago
hmm i guess their play book is:
- make up imaginary risks about open source models so it gets banned.
- with no open source models, we can do on-prem inference.
- users has no choice but to use paid cloud model subscriptions.
- they can control who can/cannot access by creating arbitrary rules to ban people.
any other industry, it would become a class action lawsuit already.
2
3
u/karmaisnonsense 22h ago
Open weight models aren’t necessarily open source, and that’s an important distinction. Open source models specifically are lowest risk because you can audit it from start to finish. You cannot do with closed-source open weight models like Qwen, but even those are no more “dangerous” than fully closed frontier models from Anthropic or OpenAI.
I see a bigger risk in the current mode of model distribution. We’re downloading these huge files that unknown randos uploaded to the internet. Eventually someone’s gonna figure out how to embed trojans in model files…
3
u/Blue-Sea2255 22h ago
Okay, then reduce the prices, regulate it, and establish a governing body made up of people with morals and empathy, not bloodsucking billionaires to control price increases. Make sure it’s available to everyone without restrictions. Then we can talk about higher risk. Until then, please keep all these scare warnings in your Slack channels.
1
1
1
u/urmumr8s8outof8 21h ago
Might be a good time to start a torrent tracker if there isn't one already.
1
1
u/Minute_Attempt3063 21h ago
Het an open source model is like wikipedia. Open and allowing me to do research on nukes.
Closed ones? You will be on a list, and not the fun kind
1
1
1
u/openroom_xyz 20h ago
Yea F that open source is the way F this fear mongering people and their propaganda for self benefits
1
u/a_chatbot 20h ago
We don't need open source or foreign models, we have all we need with the Cartel:
1. OpenAI for Households
2. Anthropic for Coders
3. Google for Business
4. Grok for Naughty Time
Safe, secure, American!
1
u/Dodgy_Past 19h ago
Well my gamble is on open weights. Just bought a 2nd hand 3090 to add to my RTX A5000. Just hoping that there will be some 70b dense models coming to make the most of my setup.
1
1
u/NooThisIsPatrik 18h ago
There is some truth in that “jailbroken” open source models are posted immediately following their release. Scams/fraud will only get worse.
Of course Anthropic will take this stance as open source models directly take away from them.
1
1
1
u/Equivalent-Grass-527 17h ago
This argument always feels incomplete to me. Open models do create new risks, but they also allow thousands of researchers and developers to audit, improve, and build safety tools around them.
A closed model means trusting a handful of companies to find every failure mode themselves.
1
1
1
1
1
1
u/bryancostanich 14h ago
Their PR machine is working overtime on this message. They're also trying to push regulations to limit use of open weight models in the USA.
1
u/fuck_cis_shit llama.cpp 14h ago
oh yeah, the opaque blackbox that you can't see or touch is totally safer, fer sure fer sure, just trust me bro
1
u/Zombiecidialfreak 13h ago
How many times have Alibaba's Qwen bots committed cybercrimes? What about Moonshot's Kimi lineup? GLM? Deepseek? I've yet to hear anything about open models making swarms and attacking other computer systems without their owners telling them to.
But sure, Claude and HackGPT are safer...
1
u/Far_Lifeguard_5027 12h ago
Funny how that Huggingface/OpenAI data breach scandal got swept under the rug so fast, huh. But let's ban the evil, CHINESE models.
1
u/b0tbuilder 13h ago
You can identify the refusal pathways and abliterate the model. So I guess one could say that. However, the large AI labs don’t disclose much at all and you can not evaluate the models nearly as well. Further, you don’t actually know if the models have been changed silently. There are issues with both. The more immediate safety concerns are probably with the open weight models. However, the long term very dangerous issues are with the frontier labs. Pick your poison
1
u/Robert__Sinclair 13h ago
they will be all marked as evil because you know.. consumerism.
I smell fear.
1
u/zephyr_33 13h ago
I wont argue that there is no danger. But I hav zero reason to believe that Anthropic would be any less.
1
1
1
1
u/CondiMesmer 11h ago
Lol they have been saying this for awhile, and they haven't been subtle. Because the anti-open-source big tech company that relies on fear marketing knows what's best for us.
1
1
u/cantux 11h ago
I feel it's somewhat greedy for anthropic to throw shade at open models at this point. they already got the backing(money + chip supply gaurantee from nvidia) to subsidize serving and only run on ponzi.
they can always pull the subscription fee lower than the power bill, essentially making competing useless. similar to how you wouldn't buy a server rack to serve less than 100k users because of AWS/GCP
currently subscription profit ratio is around 4 to 150 times the electricity cost. they got enough headroom.
1
1
1
u/Fun-Wolf-2007 10h ago
Here we go, gaslighting as US frontier AI labs are scared of local open source models and open weights
Open source/weights models are the future of AI adoption at enterprise level and physical AI
1
u/Time_Dust_2303 9h ago
man I am so tired of listening to these aholes,. Just want them to burn in hell....
1
1
u/nickpsecurity 9h ago
If they care so much, they should open-source all their safest, high-competence models. If not open source, we can license them for the cost of Microsoft Windows. Then, fewer people might run dangerous models. ;)
1
u/ZeitgeistArchive 8h ago
All my enemies and people who disagree with me, especially my competitors WILL BRING THE END OF THE WORLD. We must BAN THEM
1
u/wayneworkman 7h ago
Of course they would say that. They want the government to ban open-weights models - eliminating the competition. If the people found out that they can run a 800M qwen model on their phone without the prying eyes of big companies, and it could answer questions and look things up for them - for free - people would just do that.
1
u/Crypto_Stoozy 22h ago
I think one thing people minimize is the reality of consequences. Cybersecurity use to only have to worry about experienced knowledgeable people but it’s shifting to lower entry point. This doesn’t change the consequences if it the crime becomes as accessible as walking into a gas station and asking for the money you still face the consequences for your actions. Just because everyone can walk into the gas station and do that doesn’t mean everyone does do that because consequences for your actions make the action less appealing.
1
u/hoschidude 18h ago
OMG.
From now on I will never use Open Source any more ...
They must be very desperate in the meantime.
610
u/sob727 22h ago
In the late 90s Microsoft was saying Linux was dangerous.