The loophole Anthropic may use to get around data center bans: performing Advisor only
Posted by kaggleqrdl@reddit | LocalLLaMA | View on Reddit | 18 comments
https://claude.com/blog/the-advisor-strategy
If Anthropic does the 'advisor strategy' only, and convinces people to buy hefty GPUs and install local LLMs, they can offload their datacenter requirements to home users.
Home users end up paying for the hardware, the electric bill. Home users have mostly unfettered access to grids and Anthropic doesn't get the blame for raising power prices and drawing all that water (even though, in affect, they are still doing it.)
The upside is this could give local LLMs a much needed boost which helps support competition.
However, Anthropic is historically very anti open / local LLMs, so don't be surprised if they get around that by forcing you to install their proprietary, internet on only smaller models.
takoulseum@reddit
Interesting perspective on this. The advisor strategy does make sense from a business standpoint, but the community concerns about offloading compute costs to users are valid too.
abnormal_human@reddit
I don't think anything about that article suggests Anthropic offloading work to local LLMs that they have anything to do with.
However, if you already run Local LLMs, they may have proven out a pattern that lets you increase performance to near-Opus levels with occasional, much lower cost, Opus use. Which is very interesting for us.
kaggleqrdl@reddit (OP)
That's literally what their post is about. Lol. The labs all have this delegation model already in house. The reason they are externalizing it is because they have a compute / data center problem.
The great thing about software is you can sell it, and the user maintains it. All the labs hate having to maintain everything. They'd love to offload that onto users.
CalligrapherFar7833@reddit
You literally didnt understand anything from the article . Advisor is for sonet coding everything and asking opus when its stuck
kaggleqrdl@reddit (OP)
Again. The labs all have this delegation model already in house. They have been doing this forever. The reason they are externalizing it now is because they have a compute / data center problem.
abnormal_human@reddit
Their post is about using Opus as an advisor to Sonnet/Haiku. There is nothing about open models, local models, or work being done outside of their umbrella. They are solving their compute problem by getting more work done with lesser models hosted by Anthropic, not by compromising their whole business strategy to support the r/LocalLLaMA vision.
kaggleqrdl@reddit (OP)
Again, they already do that. All of the frontier labs do that. Delegation has been done for quite awhile. The reason they are externalizing it is because they have a compute / data center problem.
"don't be surprised if they get around that by forcing you to install their proprietary, internet on only smaller models."
Apart_Ebb_9867@reddit
that blog post doesn’t talk about them externalizing anything.
kaggleqrdl@reddit (OP)
Again, they already do that. All of the frontier labs do that. Delegation has been done for quite awhile. The blog post IS the externalization.
Webfarer@reddit
Bro is hallucinating.
Apart_Ebb_9867@reddit
Oh, talking about something they do internally is “externalization” and this is the smoking gun proving they have a data center problem? And that tells you they’ll force people to install their models on your machine? Dude ask Claude code to read and explain the blog post to you because your reading and comprehension abilities are quite limited.
cyanheads@reddit
... it's not some special "advisor" tuned model they're using. This is a technique people have already been using for the past 2 years.
kaggleqrdl@reddit (OP)
Yes, this model goes back as far as when gpt first came out. There's nothing new here, except Anthropic exposing it.
GoodTip7897@reddit
There is simply no way unless a drastic change happens that consumers can afford the hardware needed to run Claude locally.
Yes Gemma 31b and qwen 27b are amazing beasts that have replaced 95% of online model usage for coding for me, but they aren't going to entice non tech savvy users to replace Claude.
kaggleqrdl@reddit (OP)
I'm sure they can build a claude flash model that can handle a lot of low level tasks. The advisor is what matters, and no, you won't be able to run that locally.
GoodTip7897@reddit
If we're being real most consumers cannot afford a 16gb GPU let alone a 24gb one... Likely most could afford to run a 2-4B active model and that's just not going to be able to compete with any flash models online (100b+ Moe or 27B+ dense is needed from my experience to compete with online flash models for general tasks)
kaggleqrdl@reddit (OP)
People who pay anthropic are not 'most consumers'. They spend thousands of dollars per year on tokens.
Apart_Ebb_9867@reddit
what is this data center ban thing?