Local LLM vs APIs — which one ended up more practical for you?
Posted by HealthySkirt6910@reddit | LocalLLaMA | View on Reddit | 5 comments
For people who’ve tried both:
Running local models vs using APIs
Which one ended up being more practical for you?
I thought local would be cheaper, but not 100% sure anymore.
MK_L@reddit
What do you mean? Can you elaborate? How is it cheaper to use api?
HealthySkirt6910@reddit (OP)
Running local models requires significant hardware investment and comes with a high learning curve, especially if you want to access powerful model capabilities. Typically, standard hardware isn’t capable of running high-parameter models. Only very powerful hardware can handle that. However, having powerful hardware introduces another issue — electricity costs, which can be quite expensive.
ttkciar@reddit
.. unless you already have a computer. For the first two years of my LLM journey I spent nothing on LLM-specific hardware, just used what I already had.
MK_L@reddit
I started writing the same thing when he made that reply. Most people who program have decent hardware unless your a mac user ( not counting macstudio which still has its place 8n the coding world)
I went from a 3080 12gb to 3090 24gb to 5090 32gb... had nothing to do with local llm. I code/run software where this is an advantage. Running a local llm was a good fit because to me its practically free.
The electricity cost where I live is nearly unnoticeable. $0.07/kw...
His approach seems to be if someone had nothing and their whole investment is in a llm box from scratch.
namakoo1@reddit
Honestly, it’s hard to pick a winner because it totally depends on your use case! ⚖️ APIs are fantastic when you need top-tier reasoning or want to get up and running instantly. They’re basically a 'superbrain' on demand, though the costs can add up if you're processing huge amounts of data. Local LLMs, on the other hand, are great for long-term projects and bulk tasks where privacy and 'zero cost' are the priorities. They might not be as smart as the largest APIs, but they’re perfect 'workhorses' for specific, repetitive jobs. Bottom line: There’s no one-size-fits-all. It’s all about picking the right tool for the task. What kind of project are you thinking about?