That's why local models are better
Posted by Illustrious-Swim9663@reddit | LocalLLaMA | View on Reddit | 220 comments
That is why the local ones are better than the private ones in addition to this model is still expensive, I will be surprised when the US models reach an optimized price like those in China, the price reflects the optimization of the model, did you know ?
Aguxez@reddit
I'll patiently wait until I can run Opus locally
diagonali@reddit
How long before we get Opus 4.5 levels local models running on moderate level GPUs I wonder? 5 years away?
314kabinet@reddit
There was a paper that showed that any flagship cloud model is no more than 6 months ahead of what runs on a 5090, and the gap is shrinking.
Frank_JWilson@reddit
Whoever wrote the paper was high on something potent. By that logic we could be running Sonnet 3.7 or Gemini 2.5 Pro on a 5090 by now. Even the best open models aren't at that level and they aren't even close to fit on a single 5090.
Frankie_T9000@reddit
I run big models on local using CPU and video, its all good if you arent in a hurry
314kabinet@reddit
Fair, the numbers are probably off. Then again these days you can models better than the original GPT-4 can run on 64GB DDR5 with CPU only. I mean the newer Qwen MoE models. So if not 6 months then no more than 2 years and not 5 like OP suggested.
lorddumpy@reddit
tks is an issue as well. Having to retweak your prompt and wait another 30+ minutes for it to generate is not a great experience
davl3232@reddit
I guess the point being made is new open source local models with the same or similar quality become available 6 months from frontier model release. Not that you can run the exact same model locally.
CheatCodesOfLife@reddit
We have better local models today than SOTA one year ago
daniel-sousa-me@reddit
But not ones that can run on "moderate level GPUs", right?
CheatCodesOfLife@reddit
People are getting 10 t/s running Kimi/Deepseek quants with no GPU at all
throwaway2676@reddit
Depends on how long it takes an H100 to be considered a moderate level GPU
Low_Amplitude_Worlds@reddit
I cancelled Claude the day I got it. I asked it to do some deep research, the research failed but it still counted towards my limit. In the end I paid $20 for nothing, so I cancelled the plan and went back to Gemini. Their customer service bot tried to convince me that because the compute costs money it’s still valid to charge me for failed outputs. I argued that that is akin to me ordering a donut, the baker dropping it on the floor, and still expecting me to pay for it. The bot said yeah sorry but still no, so I cancelled on the spot. Never giving them money again, especially when Gemini is so good and for eveything else I use local AI.
Specter_Origin@reddit
I gave up when they dramatically cut the 20$ plans limits to upsell their max plan. I paid for openAI and Gemini and both were significantly better in terms of experience and usage limits (Infact I never was able to hit usage limits on openAI or Gemini)
Sharp-Low-8578@reddit
To be fair a huge issue is that it is not actually affordable and any affordable option is other subsidized losing money. Just because improvements in capacity are strong doesn’t mean they’re actually more accessible or reasonable cost wise, we’re far from it if they’re on track at all
Specter_Origin@reddit
In all honestly as a consumer I couldn’t care less, specially not in this economy xD
Danger_Pickle@reddit
This. As a professional software developer deploying cloud applications and running my own local models, I understand almost exactly what their costs per-request are. But as a customer, I have zero interest in paying for a product that I don't receive, and I have little interest in paying full price for something when their competitors are heavily subsidizing my costs. While the bubble is growing, I'm going to take advantage of it.
Will this inevitably lead to the AI bubble popping when all these companies need to start making a profit and everyone has to increase their API costs 10x, thus breaking the current supply/demand curve? Absolutely. Do I care? Not really. The only companies that will be hurt by the whole situation are the ones that are taking out huge debt loads to rapidly expand their data center infrastructure. The smart AI providers are shifting that financial burden onto companies like Oracle, who will eat the financial costs when the bubble pops. But I can't do anything to change those trends, so I'm not worrying about it.
Liringlass@reddit
You don’t pay for the output, but for the thing that produces the output. I don’t see how a failed output could be not payed for when we’re the ones who control the input.
It’s like renting an oven and burning your bread. The rental company won’t refund your bread.
banithree@reddit
BTW: How to avoid anthropic to put smarties on my bread?
Danger_Pickle@reddit
I love this analogy, because I do a decent amount of baking, and let me tell you there's a HUGE difference in the quality of certain ovens. My current apartment has the standard cheap apartment appliances, and the oven frequently burns things or fails to maintain a consistent temperature even brand new. I've used three ovens of the exact same model, and they're all barely usable trash. Regardless of how good my cooking is, the oven is inconsistent in how it performs.
Meanwhile, I own a nice luxury toaster, which bakes things far better than my apartment oven. It maintains temperature very well, and whatever black magic temperature/moisture sensors are in there it always cooks things perfectly with minimal effort on my part. I'll pull things out of the freezer and it'll still cook evenly in spite of a buildup of ice only on a single side. I do barely any work, and the toaster makes my life easy.
Modern APIs are total trash. They're cheap apartment ovens where you repeat exactly the same procedure but things outside your control cause problems. I've sent the same exact input to different APIs but I get errors sometimes and reasonable responses other times. I say the APIs are trash because I maintain back end APIs for production software, and my APIs have 1/1000th the error rates of anything on Openrouter. Lots of these APIs don't even have 90% reliability, let alone the 99.99% that I think is reasonable for most production applications. Modern AI APIs are less reliable than the stuff we built in the 90s, and I refuse to accept "it's complicated" as a reason because I've never had similar issues running my own models locally, and I've deployed plenty of complicated software with better reliability numbers than the current APIs.
At the end of the day, the bubble is going to pop and someone is going to have to pay for the cost of the failed requests. Businesses aren't going to lose money, so the costs will eventually be subsidized by the customer. "Free if it fails" will eventually be rolled into the cost, so an API with a 50% failure rate will necessarily cost twice as much as an API with zero failures. The whole reliability and consistency insanity (plus privacy concerns) are why I've spent less than 20$ on APIs, and I'm avoiding expensive APIs that charge me money when they break. I know exactly what it costs to run these tools, and I refuse to pay for someone else's DevOps mistakes. I have a wallet and I'm voting with it.
Few-Frosting-4213@reddit
That analogy would only work if it was user error, which is not the case a lot of the time.
Liringlass@reddit
I get you, but if you think about physical tools and take a primitive one, you might get failures even when using it right. Skill allows you to diminish but never remove that risk.
Kind of like with prompting :) good prompts get better results but good results aren’t guaranteed.
I’m not sure we’re at a stage where AI can be expected to be flawless yet :)
BarelyZen@reddit
Consolidation will happen when the bubble bursts. Just like other bubbles. There are players in the market, right now, that are loading up on debt knowing full well that they are going to offload that debt to a subsidiary/acquisition that will then be taken into bankruptcy. It's as old as the robber barons; same strategy, different sector.
Danger_Pickle@reddit
Yup. OpenAI seems like the posterchild for a massive bankruptcy, and Microsoft has carefully kept that financial disaster as a separate corporate entity so they don't have to eat the one trillion dollars of contractual obligatory expenditures. I struggle to imagine who's going to buy OpenAI. They're a financial liability and they bleed money. Oracle's stock price has already fallen 30% in the last month putting it below the huge AI price spike, so people are starting to catch on that their huge datacenter contracts with OpenAI are worthless.
My current bet on the most successful company is Anthropic. They're charging something close to the real costs of their APIs, and they're focusing on profitable corporate contracts instead of nonsense like generating ticktock videos (See: Sora). They've also got arguably the best models and they're collaborating on actual research into things like poisoning, so it's likely that they'll keep up with the pace of the rest of the industry. Their debt load is relatively small compared with their revenue, and they have an actual path to profitability. They've got a smaller percent of the market than OpenAI, but that's arguably a good thing, since they're well positioned to become dominant after the bubble pops. They're everything OpenAI isn't.
If Anthropic somehow manages to go bankrupt then this bubble is bigger than even the largest estimates, or there's so much financial fraud in the system that even well run companies are going under. I'm not worried because that would mean we've got much bigger economic problems that make the current bubble predictions look quaint.
Still, even if I'm bullish on their long term financials, I'm not paying for their API prices.
Anxious_Comparison77@reddit
It's going to be XAI and Nvidia as primary drivers, Sam Altman was snubbed at the AI meeting with Trump last week including the Saudi's. The Musk/Trump bromance is back and heck more doge cuts are expected soon.
Now they announce project genesis. Grok is by far more advanced than people realize, Grok 5 should be pushing 6 Trillion parameters around 4x of Grok4.
Also XAI datacenter is leased to own, Sam Altman has to rent everything for massive losses, and they have no robotics studies running, no self driving cars etc.
Musk has hoards of other AI related tech to go with it, like catching rockets in the air while not blowing up (usually) :)
The main loop is Trump, Musk, Jensen. It always has been.
RobotArtichoke@reddit
Open ai has invested heavily in humanoid robot company, figure ai
Danger_Pickle@reddit
We agree that Sam is doomed, but the most important advancements in AI have come from massively reducing the cost to train and run models. Our modern AI revolution was kicked off by reducing compute costs 100x with the paper Attention is all you need, and recent MoE architectures promise another \~10x reduction in the compute cost of running and training models. There are a dozen other opportunities for reducing the compute costs. That means the raw compute power matters a whole lot less than anyone realizes. That realization makes own mountains of Nvidia GPUs a lot less important. Smaller companies have a relative advantage because they aren't trying to force engineers to utilize billions of dollars of computing power just to repay their investments. Just look at Deepseek beating ChatGPT with WAY less compute because the bothered to optimize their compute costs. Owning tons of GPUs is a liability, not an advantage.
But ultimately, Grok is going to fail for reasons that have nothing to do with compute costs and GPU ownership. The real problem with Grok is the mecha-hitler problem. Grok is run by someone who's incredibly unreliable, which means it's never going to be the most successful product in a world where corporate contracts are the most important factor in profitability. Most corporations stopped running ads on Twitter because they value stability, predictability, and public image. None of Elon's companies those things, so they're never going to win enough large corporate contracts to pull ahead in the long term. I've seen companies buy IBM mainframes because IBM is reliable, predictable, and has a good sales team. The technology isn't good, but IBM makes a ton of money selling sub-par products to corporate customers who value stability over performance. That's where the real money is. Anthropic seems to understand that, while none of their competitors do. I think that's going to make the biggest difference.
The other problem with Grok is the constant Elon glazing, But hey, it's easy to turn that into a joke, so maybe it's not all bad. I bet Elon really would be the world's best poop eater. See: https://x.com/PresidentToguro/status/1991599225180971394
Sharp-Low-8578@reddit
Oh it’s not a defense! I don’t support them, they just kinda pretended to be financial viable and sucker people in. There’s NO way their models will stay safe and stay the same price. Something’s gotta give. Either their device turns to shit as it is right now or they’re selling your data. I personally wis they’d stick to research and stop polluting the economy and data center towns
AcrobaticContext@reddit
Please, don't remind me of their data mining. It's too painful for me to even think of again.
Ok-Wasabi2873@reddit
Sounds like they need to use AI to create a sustainable business model.
aeroumbria@reddit
How come? Plenty of endpoint and instance providers are running along just fine at average market price. People are still willing to pay, just not at extortion price wrapped in gacha game fatigue machnics.
Anxious_Comparison77@reddit
I been messing with it, lately. The lower tier plans are neutered to entice people to pay the $100s per month. Coding is bullshit unless you buy the expensive plan,
Internally at the data centre they are perfect coders, what you get from corporate is slop and full of propraganda.
IrisColt@reddit
As a free user of Gemini, you immediately run into limits.
IntolerantModerate@reddit
I use Gemini all day long everyday with my Google Workspace and never hit a limit.
IrisColt@reddit
I use https://gemini.google.com/app and only three prompts before blocking further requests.
IntolerantModerate@reddit
Paid, workspace, or free? I've never hit a limit and I have it doing coding in think mode a lot
IrisColt@reddit
Er... the free one.
IntolerantModerate@reddit
I'm on like a $9/month workspace plan so I get my domain email. And it comes with Gemini, so a good deal.
218-69@reddit
Untrue. Jules, 15 free 2.5 pro uses, n amount of prs possible for the repo in the session. Gemini CLI, 1000 2.5 pro requests in a day, can be plugged into any code assist with openai api reroute. Ai studio, basically infinite casual in chat use. Antigravity, currently basically no limits, or 2-5 hour time outs after 1 hour of constant requests, and can switch to claude 4.5 sonnet in the same session that can also get a bit of a work done in the downtime.
Maybe you're jacking off to fast. You can take a break sometimes and try doing other things.
IrisColt@reddit
I meant raw Google Gemini 2.5 from Google's GUI, three to five prompts and instant quarter of a day backoff time.
Specter_Origin@reddit
Yeah I am not talking about free… I am talking about their paid 20 bucks sub, for Claude for 20 bucks you can have like 40-50 messages with Gemini you have have more line 400, it’s just a ballpark btw
IrisColt@reddit
Thanks for the info!
Bakoro@reddit
As far as I can tell, OpenAI and Google don't do a hard cutoff on service the way Anthropic does.
Anthropic just says "no more service at all until your reset time", OpenAI and Google just throttle you or divert you to a cheaper model.
mister2d@reddit
I hit hard cutoffs with OpenAI all the time with my paid account using RooCode.
Bakoro@reddit
I believe that since you're using API access, and they're trying to get you to pay per million tokens.
If you hit the cap via API, do you also get cut-off from the browser chat interface? Like, not more services at all?
Just FYI, if you've got a ton of MCP servers running, that's going to eat tokens like mad. Also If you're doing complied code, make sure the compilation isn't generating millions of tokens that are being processed by the LLM, I made that mistake the first day using Claude Code, and blew through the cap almost instantly.
gunererd@reddit
Do you use a cli client for Gemini?
Ylsid@reddit
Broke: using Claude to code and always running out of limit
Woke: use the customer retention bot to process prompts
swagonflyyyy@reddit
Try paying $200 for a year days before this post as a first time user. I really threw away that money. Omg. Can't get past 2 messages without reaching the "limit" they set.
TF is going on over at Anthropic???
cl_0udcsgo@reddit
They had the bot respond like how a human they employ following company guidelines would. That's the only good thing from their side lmao.
TheRealGentlefox@reddit
Gemini 3 is now omega-SotA anyway. Hopefully LLMs will be super cheap by the time Google stops spending countless billions to subsidize it for us.
VampiroMedicado@reddit
Are API prices real? I wonder if Opus was reasonably expensive (if it had a high cost to run).
Opus 4.1 was insane 15$/75$ per 1M, now Opus 4.5 is 5$/25$ which would be easier to subsidize in theory.
smashed2bitz@reddit
You need like 8 GPUs to run a large 200B+ model... and each of those GPUs are like $20,000.
So. Yah. A $200,000 server plus the power it consumes adds up fast.
danielv123@reddit
Afaik all providers are making money at api pricing, but it's hard to tell how much. Also none of the big labs make enough to pay down the investment in model training and research.
BarelyZen@reddit
I've found Google's Vertex to be very satisfying when I need to run things that need larger context windows. I often have 6-7 free AI's open and run my brainstorming through them and turn to Vertex when I'm ready to start creating prototypes or drafts.
Background-Quote3581@reddit
Upvote for the laugh I had about the dropped donut anology.
therealAtten@reddit
WOW I had the exact same experience, the exact same argument with the bot and with a different analogy and got so pissed off as well. If you try and post that on the r/ClaudeAI your post get's instantly deleted. Haa, silencing valid criticism always backfires at the end. Thanks for speaking out of my mind!
mister2d@reddit
Not to mention that Claude mysteriously loses your data. There are times that past conversations or code can't be found.
VoltageOnTheLow@reddit
Similar experience for me. People are way too kind to Anthropic, they have oversold for their capacity, and rather than limiting sign-ups, they basically land up scamming their lower tier subscribers.
Jayden_Ha@reddit
I use open router, all failed requests doesn’t cost anything
LinkSea8324@reddit
Gemini and claude are good because it easily let you import code
Claude allows you to import code but cries as soon as there is too much lines
Gemini doesn't give a fuck and eats it all
Salty-University2744@reddit
If it wouldn't fail they'd be charging 10k instead of 20 bucks though
Low_Amplitude_Worlds@reddit
Absurd take.
Final-Rush759@reddit
They do lose money every time they serve you. I think OpenAi is already switch to more affordable models. Google has always been more conscious about the running cost. They always have their own TPUs which are much cheaper than Nvidia GPUs.
MossySendai@reddit
I just switch between free plans on all the top model providers. I prefer non-thinking models anyway.
lostnuclues@reddit
I think Anti gravity(geminie 3 pro) and codex would do that. And both are way cheaper than Anthorpic.
Bananaland_Man@reddit
Local models can barely code, especially if you don't have the vram for larger models. Not saying I suggest anyone use an llm to code at all, but comparing local models to something like Claude or deepseek is like comparing a go kart to a formula 1. (again, I don't think people should use llm's to code, they all suck, but programming is the worst thing to try to get people on board with local models for.)
ohwut@reddit
Anthropic is basically hamstrung by compute, it's unfortunate.
The other $20 tiers you can actually get things done. I keep all of them at $20 and rotate a Pro across the FoTM option. $20 Claude tier? Drop a single PDF in, ask 3 questions, hit usage limit. It's utter unusable for anything beyond a short basic chat.
JoyousGamer@reddit
I get things done on Claude just can't use their latest OPUS and 4.5 can possibly go a little too quickly as well.
Your issue is you are putting a PDF in Claude when you should be putting in the actual code. You are chewing through your limit because of your file format.
ohwut@reddit
Yet I can dump the same, and more, pdfs into literally any other consumer frontier LLM interface and have an actionable chat for a long period. Grok? Gemini? OpenAI? I don’t need to complicate my workflow, “it just works”
This comment is so “you’re holding it wrong” and frankly insulting. If they don’t want to make an easy to use consumer product, they shouldn’t be trying to make one. Asking grandma “oh just OCR your pdf and convert it to XYZ” before you upload is just plain dumb.
JoyousGamer@reddit
Okay but Claude is for coding not asking how to make friends.
Be upset though and use tools wrong if you want it doesn't impact me. I thought I would help you out.
ohwut@reddit
“ClAudE iS fOr CoDiNg”
K. Why do they have a web app, mobile app, and spend millions advertising all the non-coding things it can do? Open your mind man.
If Claude is for code, they would just have an API and Claude Code.
SlowFail2433@reddit
Google wins on compute
cafedude@reddit
And they're not competing for GPUs since they use their own TPUs which are likely a lot cheaper for the same amount of inference-capability.
daniel-sousa-me@reddit
Well, sort of
The bottleneck is on the manufacturing and afaik they're all dependent on the capacity of TSMC and ASML
SlowFail2433@reddit
Yeah around half as cheap according to a recent analysis
randombsname1@reddit
Good thing is that they've just signed like $100 billion in deals for massive amounts of compute within the last 4-6 months.
314kabinet@reddit
Hell I get things done on the $10 tier with Github Copilot.
yungfishstick@reddit
This is pretty much why I dropped Claude and went mostly local+Gemini for everything else. Personally, I don't care how good your LLM is if I can barely use it even after paying for a paid tier
rubba_tt@reddit
What are the best local ones to use
burntoutdev8291@reddit
Use the cheapest GLM, i have never hit my limit once
PiotreksMusztarda@reddit
You can’t run those big models locally
yami_no_ko@reddit
My machine was like $400 (Minipc + 64 gb DDR4 RAM). It does just fine for Qwen 30b A3B at q8. Not the fastest thing you can get, but its enough for coding given that it doesn't run into token limits.
Here's what I've made based on the system using Qwen30b A3B:
This is a raycast engine running in the Terminal utilizing only ascii and escape sequences with no external libs, in C.
MackenzieRaveup@reddit
Absolute madlad.
pureroganjosh@reddit
Yeah this guy fucks. Absolutely insane but low key fascinated by the tekkers.
yami_no_ko@reddit
Map and wall patterns are dynamically generated at runtime using (x \^ y) % 9
Qwen30b was quite a help with this.
peppaz@reddit
Thanks for the cool fun idea. I created a terminal visualizer base in about 10 minutes with Qwen3-coder-30b. Am getting 150 tokens per second on a 7900XT. Incredibly fast and quality code.
Check it
https://github.com/Cyberpunk69420/Terminal-Visualizer-Base---Python/tree/main
Novel-Mechanic3448@reddit
Who are you responding to? that has nothing to do with the post you replied to
HornyGooner4401@reddit
I think "you don't need big model" is the perfect response to "you can't run big models"
Claude's quota limit is ridiculously low considering there are now open models that matches like 80% Claude's performance for a fraction of the price that you could just re-run until you get your expected result
Maximum-Wishbone5616@reddit
Kimi k2 crush the claude sometimes by 170% in tests. IRL not even close for real work. So who cares about some 2024 hosted models if you can run qwen3 that do exactly what devs need, ASSIST. AI freely generated model is a hell to manage, plus you cannot copyright, sell it, get investors or grow. What is the point? To create an app for friends??? You employees can copy entiet codebase and use it as they wish!
1Soundwave3@reddit
Who told you you can't copyright or sell it? Nobody fucking cares. Everybody is using AI for their commercial products. It's even mandated in a lot of places.
yami_no_ko@reddit
I've responded to the statement
LarsinDayz@reddit
But is it as good? Nobody said you can't code on local models, but if you think the performance will be comparable you're delusional.
yami_no_ko@reddit
Wasn't telling that. Sure, there's no need to discuss that cloud models running in data centers are more capable by magnitudes.
But local models aren't as useless and/or impractical as many people imply. Their advantages make them the better deal for me, even without an expensive rig.
Maximum-Wishbone5616@reddit
Kimi k2 wiped the floor with opus/sonnet.
Today's CC Sonnet is just horrible at work. It cannot just simply follow existing patterns in a codebase. It always changing and mixing. can CC create some fun stuff out of nothing in 20minutes? Sure better than qwen. But that not what you need in enterprise level platform serving millions requests every day. I just need an assistant that quickly create new views, use existing pattern for new entities and this it. Create sql statements etc.
No AI can replace dev, but it can boost a productivity. CC is horrible as a code monkey, and I already know much better how to create large scale platform, I do not need silly games or other silly showcase how great CC can be, as it is not its use case. It is to save money and make more money. When you deploy LLM for 40 deva you need local, fast, and predictable output.
Maximum-Wishbone5616@reddit
? It is much better irl. It does follow instructions and just follow existing pattern. I decide what patterns I use, not half brain dead ai that cannot remember 4 classes back. CC is horrible due to introducing huge amount of noise. super slow, expensive and just bad as assistant for a senilr.
SkyFeistyLlama8@reddit
Qwen 30B is surprisingly good if you keep it restricted to individual functions. I find Devstral to be better at overall architecture. The fact that these smaller models can now be used as workable coding assistants just blows my mind.
dhanar10@reddit
Curious question: can you give more detailed specs of your $400 mini pc?
yami_no_ko@reddit
it's a AMD Ryzen 7 5700U MiniPC running on CPU inference(llama.cpp) with 64GB DDR4 at 3200 MT/s
noiserr@reddit
So I gotta question for you. Do you find running at Q8 as opposed a more aggressive quant noticeably better?
I've been running 5-bit quants wonder if I should try Q8.
yami_no_ko@reddit
I use both quants, depending on what I need. For coding itself I'm using Q8, but also Q6 works and is practically not distinguishable.
Q8 is noticably better, but if you're giving it easy tasks such as analyzing and improving single fuctions Q4 also does a good job. With Q5 you're well within good usability for both, coding, refactoring as well as discussing the concepts behind your code.
If your code is more complex go with Q6\~8, but for small tasks within single fuctions and discussing Q4 is perfectly fine.
noiserr@reddit
Will give Q8 a try. When using OpenCode coding agent Qwen3-Coder-30B does better than my other models but it still makes mistakes. So will see if Q8 helps. Thanks!
a_beautiful_rhind@reddit
ahh yes. qwen 30b is absolutely equivalent to opus.
Intrepid00@reddit
You can if you’re rich enough.
muntaxitome@reddit
welll... a 200k machine will allow you to purchase a claude max $200 plan for a fair number of months... which would allow you to do much more use of opus.
teleprint-me@reddit
I once thought that was true, but now understand that it isnt.
More like 20k to 40k at most depending on the hardware if all youre doing is inferencing and fine tuning.
We should know by now that the size of the model doesnt necessarily translate to performance and ability.
I wouldnt be surprised if model sizes began converging towards a sweet spot (assuming it hasnt already).
CuriouslyCultured@reddit
Word on the street is that Gemini 3 is quite large. Estimates are that previous frontier models were ~2T, so a 5T model isn't outside the realm of possibility. I doubt that scaling will be the way things go long term but it seems to still be working, even if there's some secret sauce involved that OAI missed with GPT4.5.
zipzag@reddit
The SOTA models must be somewhat MOE if they are that big
CuriouslyCultured@reddit
I'm sure all frontier labs are on MoE on this point, I wouldn't be surprised if they're ~200-400b active.
smithy_dll@reddit
Models will become more specialised before converging as AGI. Google needs a lot of general knowledge to generate AI search summaries. Coding needs a lot of context, domain specific knowledge.
eli_pizza@reddit
Is Claude even offered on-prem?
Intrepid00@reddit
Most of the premium models are cloud only because they want to protect the model.
a_beautiful_rhind@reddit
I thought only thru AWS.
Howdareme9@reddit
There is no local equivalent of opus 4.5
Danger_Pickle@reddit
This depends on what you're doing. If you're using Claude for coding, last year's models are within the 80/20 rule, meaning you can get mostly-comparable performance without needing to lock yourself into an ecosystem you can't control. No matter how good Opus is, it still can't handle certain problems, so your traditional processes can handle the edge cases where Claude fails. I'd argue there's a ton of value in having a consistent workflow that doesn't depend on constantly having to re-adjust your tools and processes to fix whatever weird issues happen when one of the big providers subtly change their API.
While it's technically true that there's no direct competitor to Opus, I'll draw the analogy of desktop CPUs. Yes, I theoretically could run a 64 core Threadripper, but for 1/10th the cost I can get an acceptable level of performance from a normal Ryzen CPU, without all the trouble that comes with making sure my esoteric motherboard receives USB driver updates for peripherals I'm using. Yes, it means waiting a bit longer to compile things, but it also means I'm saving thousands and thousands of dollars by moving a little bit down on the performance chart, while getting a lot of advantages that don't show up on a benchmark. (Like being able to troubleshoot my own hardware and being able to pick up emergency replacement parts locally without needing to ship hard to find parts across the country.)
RedShiftedTime@reddit
There will be, which is why if you want to run these in the future, local is still better.
pigeon57434@reddit
ya maybe in like 8 months the best you can get open source today assuming you can somehow run 1t param models locally is only about as good as gemini 2.5 pro accross the board
LandRecent9365@reddit
Why is this downvoted
Bob_Fancy@reddit
Because it adds nothing to the conversation, of course there will be something eventually.
redditorialy_retard@reddit
10 x H200s:
zhambe@reddit
No kidding, right? I've got a decent setup at home, but I still shell out for Claude Code, because it's simply more capable, and that makes it worth it. Homelab is a hedge and a long-term wager that models will continue to improve, eventually fitting an equivalent of Sonnet 4.5 in < 50GB VRAM
zipzag@reddit
With current trends, in the future, a Sonnet equivalent will probably fit in that much VRAM. But the question is if you will be satisfied with that level of performance in two or three years. At least for work functions.
For personal stuff having a highly capable AI at home will be great. I would love to put all my personal documents into NotebookLM. But I'm not giving all that to google.
Trojan_Horse_of_Fate@reddit
Yeah, there are certain things that I use my local models for, but it cannot compete with a frontier model
Lissanro@reddit
I run Kimi K2 locally as my daily driver, that is 1T model. I can also run Kimi K2 Thinking, even though in Roo Code its support is not very good yet.
That said, Claude 4.5 Opus is likely is even larger model, but without knowing exact parameter count including active parameters... but in any case, future models may be better optimized, so it is entirely possible that by the next year 1T open weight models will be even smarter than today's Claude 4.5 Opus.
dairypharmer@reddit
How do you run k2 locally? Do you have crazy hardware?
Lissanro@reddit
EPYC 7763 + 1 TB RAM + 96 GB VRAM. I run using ik_llama.cpp (I shared details here how to build and set it up along with my performance for those who interested in details).
The cost at the beginning of this year when I bought was pretty good - around $100 for each 3200 MHz 64 GB module (which is the fastest RAM option for EPYC 7763), sixteen in total. Aprroximately $1000 for CPU, and about $800 for the Gigabyte MZ32-AR1-rev-30 motherboard. GPUs and PSUs I took from my previous rig.
daniel-sousa-me@reddit
So the hardware alone costs like 5 years of the max 20x plan? Plus however much electricity To run a worse model at crawling speed 🤔
Don't get me wrong, I'm a tinkerer and I'm completely envious of your setup, but it really doesn't compete with Claude, which is by far the most expensive of all providers
Lissanro@reddit
You are making a lot of assumptions. Claude subscription is not useful for working in Blender, which also heavily utilizes four GPUs, and doing many other things not related to LLMs but requiring high RAM. So, it is not just for LLMs in my case. Also, I earn using my rig more than it costs - since freelancing using my PC is my only source of income, I think I am good.
Also, the models I run are the best open weight models and are not "worse" for my use cases, and have many advantages that are important to. Cloud models can also offer their own advantage for different use cases, but they have many disadvantages also.
Speed for me is good enough - often the result, even sometimes with additional iterations and refinement, gets completed before I manage to write the next prompt or was working on something else. Faster LLM would not save me much time. But of course depends on use case, for vibe coding which relies on short prompts and a lot of iterations maybe it would be slow. As of bulk processing some simple tasks, for that I can run smaller fast models when required.
But I find big models is much better at following long, detailed prompts that do not leave much wiggle room for guessing (so in theory any smart enough LLMs would produce very similar result), but increase productivity by many times because I don't have type manually most of boiler plate stuff or look up small details about syntax, etc.
In terms of electricity, running locally is cheaper last time I checked, even more so if using cache a lot - I can return even to few weeks long chat immediately without processing again, so the cost practically zero for input tokens, the same is true for reusing long prompts.
In any case, it is not just about cost saving for me... I would not be able to use cloud. Lack of privacy, cannot send most of projects I work on to a third-party and would not send my personal stuff either, cannot use cloud GPUs in Blender for real-time modeling and lighting, or any other work requiring having them physically.
Finally, there is psychological factor: if I have hardware that I am invested in, I am highly motivated to put it to good use, but if I paid for rented hardware or subscription, I would have ended up using it only as last resort, even if the privacy issue did not exist and there was no limitations about sending to the third-party. This is even more important if my work depends on it - I do not want to feel demotivated or distracted by token usage costs, breaking legal requirements or filtering out sensitive private information. Like other things, it can be different for somebody else. But for me cloud LLMs just not a viable option.
Maximus-CZ@reddit
Cool, how many t/s at what contexts?
Lissanro@reddit
Prompt processing 100-150 tokens/s, token generation 8 tokens/s. Context size is 128K at Q8 if I also fit four full layers in VRAM. Or I can fit full 256K context and common expert tensors in VRAM instead, but then speed is about 7.5 tokens/s. As context fills it gets reduced, may become 5-6 tokens as it gets closer to the 128K mark.
BoshBoyBinton@reddit
Nothing much, just a terabyte of ram /s
thrownawaymane@reddit
3 months ago this was somewhat obtainable :(
DrDalenQuaice@reddit
How do I find out what the best model I can run locally is?
PiotreksMusztarda@reddit
There’s calculators online that take an LLM model, its quant, and your hardware specs (might be just gpu not sure) and it will tell you if the model will run fully in gpu / partially offloaded to ram / won’t work at all
DrDalenQuaice@reddit
Do you have a link for such thing?
relmny@reddit
How big is that nodel? how do you know?
segmond@reddit
Who is you? There are thousands of people running huge models locally.
nntb@reddit
Oh I get it you're promoting hord or other distributed via community shared resources in my correct
-dysangel-@reddit
how do you know?
mjTheThird@reddit
YOu never want to rent your slaves. You want to OWN you slaves like a true capitalists!
RollingTrain@reddit
Yes because if there's one thing communists never set up, it's slave camps, errr, I mean work camps.
mjTheThird@reddit
And now you can! With an easy payment of few RTX 6000. You too can setup your ~~work camps~~ I mean, computer clusters. To run the localLLMs to do whatever, you want.
yanyosuten@reddit
Just remember, today is as cheap as it will ever be!
jeffwadsworth@reddit
This tale is as old as the universe by now. But, I heard it was better than before. Just haven't bothered to go back to Claude yet. Loving my local KIMI K2 Thinking too much.
johannes_bertens@reddit
I'm using local + AI Foundry on Azure + GLM 4.6 coding from z_ai
Works out fine for me. Going to probably get Factory_ai as well as I'm loving Droid.
Ancient-University89@reddit
This was my experience too, and it seems to waste context habitually. Like I'd ask it to implement a feature by modifying a couple files, it'll plan the feature change in a document. Then it'll begin implementing the feature in the first file, it notices its context is filling up and begins "sundowning" and documents its progress in another markdown document. I ask if you finish off at least the current file, so it adds one more line, re reads both documents it made. Updated them, then decided to write another third document detailing it's progress. Realizing I should start a new chat I do so, and point it at one of the documents for tracking it's progress, you bet instead of trusting the document and simply continuing where the previous agent left off, it rereads and verifies the changes, notices there incomplete, and writes a fourth document now to track whats missing. If I'm lucky it now finishes off the changes in the first file, but usually it'll 'give up' noticing complex changes are requested but it's context limit is already full so it creates a tracking document for the agent in the next chat session to ignore and/or poison it's context with.
Switched to codex 5.1 and it's so much better, stays on task, doesn't blow up its context on pointless stuff, isn't annoyingly verbose or overly confident and prioritized exploring the codebase and understanding it before making changes. Like sonnet 4.5 will constantly "Perfect I found the bug it's X... Wait actually" like a dozen times, making a small change each time, none of which actually fixed the issue I described or allowed the tests to pass. I really don't understand what happened from sonnet 4, to 4.5, like it got smarter but also much less actually useful, it's context window awareness seems to just make it compelled to spend the last half of its context window doing nothing but writing the most verbose disorganized documentation possible, and manually fixing it instead of using the linting auto fix tools. I tried Opus once and hit the limits almost immediately, I think it started a project plan and failed due to the daily limit about 1/3 of the way through.
It really gives the impression of an incompetent, used car salesman of a developer. Like a completely shameless yes man who has no concept of objective reality. The amount of guidance necessary to get it to write code first, then after tests pass, quality checks pass, and I give approval, document it's work was insane and never once worked 100% reliably. The documentation it did make was excessively verbose and wasteful of tokens, I'd have to edit it or the next chat session would get blown up immediately just by reading the document to figure out where to start.
I swear I once saw Sonnet 4.5 make five different multi hundred line markdown docs to track the implementation of a simple feature, of which it's only added about 10 lines of code, and run none of the quality checks for. Then it gets confused because the tests say it doesn't work but the docs (that it crapped out) say it should work.
It's super weird because sonnet 4 did not have this problem and it used to be my go to coding llm, and neither have any of the chatgpt codex models. Something about sonnet 4.5 makes it simultaneously once of the smartest (excluding chatgpt codex 5/5.1) and one of the absolute dumbest coding agents. It doesn't surprise me that Opus 4.5 would be similar, just dumber at a much larger scale.
JoyousGamer@reddit
Did you tell it to stop? Direct it to not be tracking all the documentation and explain everything in technically. You can strip it down to just get the code. You can also just ask for the updated sections as well instead of a whole file.
JoyousGamer@reddit
Not sure anything would touch Claude for coding locally unless you are doing something tiny and need minimal help.
Also what China model is doing OPUS level stuff? Isn't the whole thing with OPUS is its the best thing around so it chews through compute right now more quickly.
Late-Assignment8482@reddit
I get that all these companies are doing things unsustainably and we're facing a cliff where they have to charge what it costs.
But Anthropic ESPECIALLY seems to be doing price hikes to test the waters on finding the "f*ck, it's cheaper to do it myself" threshold.
The sooner a user is second guessing tokens and limits, the sooner they'll do one or more of:
Blork39@reddit
To be fair, Opus 4.5's standard context length is 200k. That's a lot more than I can manage with my local setup, I get about 50k tokens on my 16GB card with an 8b-Q8.0 model. And that's with context also quantised to 8.0. Also, when I use that much it takes minutes to first token (normally it's lightning fast). And yes it's still GPU only, I checked.
For coding there's a justification for cloud IMO. I just would never put any personal data into it.
Many_Consideration86@reddit
The current models and agentic/manual workflows generate a lot of tokens which are a waste.
The economy of tokens is such that the more they generate the more they hope to get paid. So it is out of control, specially on code generation models.
On top of that most of the automated model requests end up being dead ends which don't feed into the product/query/code.
Aggravating-Age-1858@reddit
that and no fucking server issues (or if you have any its yOUR fault lol)
more powerful and more affordable ai hardware is really the way to go
lately nano banana pro is driving me f-cking crazy. sure its the most powerful ai image tools ever made BUT the servers are absolutely FUCKED Up at the moment. but its so damn good your wilin to sit through the damn frustration even when now like every other generation fails or more if it was local. no issues. . no stopping your momentum with your awesome new ai project because the damn servers decided to conk out on you midway through local LLM is really the way to go
now if we can just have a image gen that is as powerful as nano pro AND local lol
someday! just not today lol
Astorax@reddit
I don't have the power to run such a model locally 🫠
WiggyWamWamm@reddit
Claude’s output is definitely better but the usage limits are so strict. There are many ways to make the limit last much longer. I periodically take everything we’ve made and a summary generated by Claude and start it in a new chat.
Next_Sector_1548@reddit
local control, no quotas, no mystery limits, yes this is the way.
Hyphonical@reddit
Ah yes, some 7B local model running on a nintendo DS's hardware is better than a 700B model running on a professional data center.
Not everyone has unlimited ram to store the context window.
carnyzzle@reddit
Still would rather run a local version of GLM 4.5 Air or pay Openrouter for models like DeepSeek or Kimi K2 and save tons of money that way
Hyphonical@reddit
I do use openrouter, it's great, cheap, fast and easy. But you can't compare some mistral 7B model to claude 4.5.
SimplyRemainUnseen@reddit
Yeah a small model that you OWN is better than some "safety" aligned cloud model that chugs electricity to produce marginally better output
Luston03@reddit
Which 7b model it's? Lol
FriendlyStory7@reddit
I had the same experience a while ago. I paid the $20, barely used it, hit the limit, proceeded to unsubscribe, and went back to ChatGPT and Gemini.
Liringlass@reddit
To be fair opus is extremely expensive. Sonnet can be used for longer, and even the small Haiku is super good.
I love local Ai but there is no way for me to run anything half as good as haiku. And if i run it on runpod the 20$ will be reached so quickly i wont last a single day compared to the month of claude.
If some benefactor gave me a machine that runs GLM 4.6 or even the Air version sure i would abandon claude.
Trojan_Horse_of_Fate@reddit
I mean, I disagree that this is why local models are better because if I tried to get my GPU to compute that, it probably wouldn't if it spent the entire month chugging
RabbitEater2@reddit
Just use the API as thats thousands of dollars cheaper than running something like opus 4.5 on local hardware. For the model of opus size, 20$ isnt much to be honest.
Maximum-Wishbone5616@reddit
well when my 2x 5090 fix claude code bugs it is time to move on. Even qwen3 code often is good enough to assist with most common time wasters. CC always was doing some random stuff on its own.
With Kimi K2 it is done deal.
I use probably 1-2M tokens easily and that does not include all content that is send back and forth ti my local llms.
Use many different ones on my dev machine.
Issue solved by one of those LLM often in 10minutes would exhaust my 6h limit (coder is much faster in t/s than cc so in 10min it generates much more text).
does not remove a single dot from 1500-2000 lines of code, yet still can do whatever I want to save my time. I do not want it to do some creative work, just copy and paste my patterns and apply to new entities. Plus loads of html/js/css. never going back.
My business also deploying new LLM servers almost every week now. We get 95-98% margin on all our services. OpenAi or antrophic api? Maybe 1-2% but we would never be able to compete for customers with their prices. Plus we have full control.
Adventurous-Date9971@reddit
Main point: self-hosted wins come from high GPU utilization and simple ops, not just model choice.
What’s your serving stack? vLLM or SGLang with continuous batching and paged KV cache will keep 5090s >70% busy; speculative decoding (small helper model) speeds code tasks a lot. For codegen, return diffs/patches only and cap max new tokens per call so you don’t waste context traffic. If quality dips with 4-bit, try FP8 or 8-bit weights with BF16 activations; Qwen-Coder holds up well there. Track power and depreciation per GPU-hour in your pricing; autosleep idle models and shard big contexts with RAG so you aren’t paying for long prompts. BYOC is great for enterprise: let them supply keys/hardware; you manage routing and guardrails.
We’ve used Kong for quotas and Keycloak for auth; DreamFactory gave us quick DB-backed REST endpoints so models don’t need schema dumps and we cut token chatter.
Bottom line: keep GPUs hot and the pipeline boring to keep those margins.
Living_Director_1454@reddit
Me : hi , how are you doing
Opus 4.5 : error.
Honkytonkidiot@reddit
I use Claude for coding Arduino and Python from my experience it's really good. I used Gemini first and it couldn't even write correct code for its own nanobanana api.. probably better now since V3 though.
Biggest_Cans@reddit
The actual solution for this type of workload is Gemini or Grok.
Michaeli_Starky@reddit
Local models are better, if you invested at very least 10 grand into hardware. And even then it's highly questionable.
Vibraniumguy@reddit
Nah just load $20 into openrouter and use whatever model you want. Even for chat gpt 5 with hours of asking questions back and forth I only used like $2. Plus you can use the openrouter API to connect to cline and code with it.
Never pay subscription fees. Use free Grok 4 for internet stuff and OpenRouter for higher reasoning/trying out new models that are cheaper. Local models are great but ultimately a backup since they arent as smart as the big models provided by these companies (unless you have a setup like pewdiepie worth like $10k lol)
WithoutReason1729@reddit
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.
alphatrad@reddit
The skill issues in this thread are entertaining. I've been on the MAX plan for most of the year, been worth every penny, never miss a beat or hit limits. Shipping production code on 20k+ line projects for clients. Thing pays for itself.
Most local models don't come close.
AizenSousuke92@reddit
which tools do you use it with?
Low_Amplitude_Worlds@reddit
Either incorrectly or disingenuously confuses the Max plan with the Pro plan then says it's a skill issue. Hilarious. Yes, I have no doubt your $200 a month plan outperforms the $20 a month plan. Really not hard to do when the $20 a month plan is worse than useless.
alphatrad@reddit
I'm sorry I was rude.
I've just seen a lot of guys who are unaware of how the context window works and blow through usage VERY FAST. There are guys on X somehow blowing through the MAX plan too. And I really do think adjusting how you prompt and work with context and caching and stuff that can help.
Also here's a suggestion; there is a GitHub project called Claude-Monitor that is great. It will tell you your current tokens, cost, time to reset, etc.
I am not sure about the lower plan, I was on it. But the MAX does have limits. It just kicks you down a notch.
But what do I know. I'm just a jerkoff on the internet. ¯_(ツ)_/¯
alphatrad@reddit
Great example, most don't know their MCP's that they loaded up are eating context sitting there.
Mine all active, are consuming 41.5k tokens (20.8%) just by being enabled - that's the cost of their schemas/descriptions sitting in context and not even from using them!!!
This stuff applies to local LLM's too. Just you'll never get rate limited. But you can send WAY more into the context window that isn't your work then some people are aware of.
Understanding this can improve your use of the tools.
dadnothere@reddit
Yes, my friend, paying for Max will always be better than buying locally... But that's the difference: you pay a monthly fee versus not paying because it runs on your hardware.
alphatrad@reddit
I use local models too. But I don't think they're near as good. Like at all. This is just a reality of how much you can actually run with the hardware you got unless you wanna dump some serious cash into building a real AI rig with more than one card in it.
Or buy a Mac Studio Ultra and be ok with slower tps
saltyrookieplayer@reddit
A single 5090 can buy you at least 2 years of Claude Max and you can't even run SOTA open models on it
Equivalent_Bat_3941@reddit
so true i was working on angular project and i ask claude to create a web component and i will verify it manually. after executing the create component command in va code it ran atleast 10 diff terminal command to verify the file it created in ide and is selected file for context in chat interface.
ai is getting ridiculous every day and just trying to be cash machine by simply consuming more token and not do actual work
AdministrativeBlock0@reddit
I've built a few three.js games using the $20 plan. I've hit the weekly limit once at the start. Since then I've started using a plan-first approach with a decent AGENTS.md file and I've never hit the limit again.
The free plans probably won't do enough to be useful but after that if you're careful the quotas seem pretty generous, especially with newer more efficient models.
galewolf@reddit
I'm curious -- did you ask it to build the whole game, or how did that go?
I've asked LLMs for help coding a feature in a game, but never the whole thing.
AdministrativeBlock0@reddit
I tried that at the start. It tries its best, and arguably it 'succeeds' in the sense that it can get some working code that sort of does what I asked for, but there are usually things that aren't what I actually wanted or performance problems. I've moved on to a much more detailed plan->refine->implement loop now.
With a detailed enough prompt and instructions files I reckon it could be done though. Just not by me. :)
LoaderD@reddit
Try running Opus 4.5 once on any non-trivial task.
I asked 4.1 to replicate something that's ~250 lines of code. It spun for a few minutes, then told me I was out of tokens for the rest of the day, even though I hadn't run any queries against their models.
rz2000@reddit
To be fair my local LLM is definitely not able to create new app to 3D model room for redecoration in a reasonable amount of time.
np-nam@reddit
this is hilarious. $20 a month is like $1 per daily usage. opus 4.5 is like $5/1M token in $25/1M token out in api usage. guess how many tokens you can emit before it surpasses the cost of using api? nobody would use the api service if you can freely use opus 4.5 with your $20 tier.
lukewhale@reddit
$20 Claude tier is not good enough for serious Claude code or opus work.
Low_Amplitude_Worlds@reddit
That's the problem, when the other providers' $20 tiers are totally good enough.
BootyMcStuffins@reddit
Yeah the $20 plan is for people chatting with the app. Not for doing any actual work
emain_macha@reddit
You can't use Opus 4.5 on the $20 plan. It can only be used on the $90 plan.
yamibae@reddit
The cost to run it locally just doesn’t make sense with current pricing, until something cheaper and specialised comes in the upfront cost is too prohibitive for a barely functional version incomparable to SOTA, you’d really be better off having 2x max 200 subs
Alkanphel666@reddit
Yeah Claude seems ridiculously expensive compared to most other models I've tried.
The_7_Bit_RAM@reddit
I use the free model, and there, I could only chat about 3-5 times in Sonnet and then reach the limit. But most of my work is done using Haiku.
kiwibonga@reddit
I cancelled because of false advertising. Website says the plan lets you use API calls but uh... No it doesn't. It grants you the privilege to find out that an additional purchase is required and you get zero API calls for free.
Aggressive-Bother470@reddit
What's the context limit?
Kako05@reddit
Pro cloude plans (20$) didn't get any boost to usage limits for opus 4.5 right?
Dummy_Owl@reddit
I dont get it. I can code up a storm in cursor for the price of a couple coffees a month. Both hobby projects and large scale enterprise environment. What do y'all do with your context that you're hitting limits?
dolche93@reddit
It's not coding, but creative writing gets really context heavy. It's very, very easy for me to want to throw in 50k tokens.
I generally get by with 20k per prompt instead, but I'd love if it I could run ~150k. Then I'd be able to include the entire book as context.
Dummy_Owl@reddit
That's fair, I think for creative writing its a lot better to go with something like NanoGPT - just run prompts through the subscription models and see if its enough. If not, then use paid ones. The subscription is like 8 bucks a month, if money is a constraint, then there is just no better deal. Local is great, but you can't get kimi k2 or glm locally, especially at good speed or at such low price. Still, I think OP is trying to code and this whole "i clicked a couple buttons and hit the limit" notion is just bizarre to me, I dont know how I'd do it even if I tried. Maybe if I gave it a full architecture document and made it go until not a single error remains and every feature is complete with tests and such? But that's just...not optimal.
dolche93@reddit
People try to do the same thing with writing. They want an entire book spit out with a 500 token prompt. They force it to write thousands of words and get surprised when they aren't allowed tens of thousands of tokens every few hours on free services.
Saffie91@reddit
I mean you re not dumb enough to ask it "make me an entire app from scratch"
NeverEnPassant@reddit
Local models are not cost effective.
SomeGuy20257@reddit
Sonnet is far superior, I stopped 30 minutes in using Opus because it acted like a fresh grad junior.
candreacchio@reddit
The $20 plan isn't really aimed at doing coding work. It's enough to wet your appetite and see the potential... The $100 plan is the minimum for any serious coding work.
And that $100 a month, pays itself back in an hour or two of dev work.
pier4r@reddit
It is undeniable that slowly prices are rising. 12 months ago with the first tier premium one could do more (in terms of tokens spent per day). Now one can do less. Sure, one can argue "the quality has risen", but the cost per token has too (if one is not going to use the APIs). This at least with claude and other compute limited vendors.
a_beautiful_rhind@reddit
Free inference definitely scaled back this year.
SlowFail2433@reddit
A year ago best model was O1-Preview which got about half the SWE-bench score that the modern models get, but SWE-bench is exponentially difficult so double score is dramatically better
candreacchio@reddit
Yes and no.
Have a look at 6 months ago. Usage for Opus 4 was very limited with the $100 a plan.
Today... Opus 4.5 has the same usage limits as sonnet 4.5, and the direct API costs have plummeted aswell... On their website
Opus 4.1
Input - $15 / MTok
Output - $75 / MTok
Opus 4.5
Input - $5 / MTok
Output - $25 / MTok
bigh-aus@reddit
This is the true issue for both users wanting to use the big models. This is partially why i think there's a bubble for this kind of stuff. They're massively discounting the cost to run for individuals. For businesses that have much larger budgets, that helps bridge the gap..
The question is are the local models good enough to run, with enough parameters? I would really like to see more specific local coding models - eg separate them by coding language - python, rust, go, C++. switch languages, switch models (and have more specialized parameters).
I tried to vibe code something in rust using qwen 30b and after two prompts the model started suggesting python code :(
Important_Bill7454@reddit
That’s why you need be an outlier AI trainer. You get the playground where you can use every advanced model for free and you never reach the limits
OkPride6601@reddit
Ya but open weights is so behind now, not to mention you can’t even remotely run any capable model locally without insanely expensive hardware.
Fun-Wolf-2007@reddit
You could also use Qwen3-Coder 480b I use it via Ollama cloud and it is for free Many times when Claude got going in circles, I asked Qwen3 to fix it and resolved the issues very quickly
ArtfulGenie69@reddit
Bahahah, they switched cursor on me once to their new and "improved" pricing model instead of the legacy point system and the same kind of thing happened to me. Luckily I had a 5$ limit and it was close to the end of the billing cycle but in just a few prompts (that it fucked up btw) it burned everything that was left and the $5 extra limit. That was just Claude sonnet too. It just uses two points in legacy mode but there is such a weird pricing thing on Claude as it is, it blows my mind really how bad it is.
If you read into it when you start using their model they start some kind of time period that is some random number of minutes and you only get like 40 of these periods in a month or something dumb. Using anything more than the time in the period is automatically charging you another period. Capitalist wet dream for sure.
coloradical5280@reddit
This tweet is either fake, or they didn't update their claude desktop or something, though i'm not sure how you could use 4.5 without an update. Anthropic rolled out compaction the day before they rolled out 4.5 opus, obviously a calculated timing.
tl;dr claude doesn't have "you've hit the context limit of this conversation" anymore.
iamthewhatt@reddit
Have you used it since the update? They absolutely still have the limits. They still have the session progression bar as well, and if you reach the end, it will 100% show you that exact same prompt from before.
But to be clear, I still think OP is fake. I routinely hit the limits within like an hour every 5 hour session limit, and I am using 100% Opus and have not yet hit the limit. So it is absolutely better than before.
coloradical5280@reddit
Limits and context window end are not the same thing. But rollouts are progressive and don’t all happen at once. I suppose it’s possible some people don’t have context window compaction and also have opus 4.5.
AntisocialTomcat@reddit
Same here. I haven't been able to finish a session with Opus, hitting continue every 4h until I called it quit. This model is dead to me, it's like it has never been released in the first place.
opi098514@reddit
Wait why would you use opus for something that trivial? Sonnet will work just fine.
eli_pizza@reddit
I don’t totally follow. You can obviously do it with Claude if you spend more? And you’d have to spend a small fortune on hardware to be able to run a local model even in the same ballpark.
No_Training9444@reddit
But everyone has different needs. I for one covet the best capabilities there are.