Every "wow look how amazing this model is" video I've seen online has given a dishonest impression aimed at engagement with naive viewers and the things demonstrated have little or no real-world value. If you type a one-shot prompt and get some slop out the other end, everyone else can also do this - you've gained nothing and lost time. The code you'll generate is basically useless no matter how good it appears. If you want to make something useful, you need to build a project properly: small steps, architected properly.

The capability of local models is now quite good, it's probably on par with the best online models from 4-6 months ago, but AI is moving very fast and 4-6 months does make a big difference. However, in my opinion the importance of objective LLM "intelligence" has begun to diminish in favor of how effective your usage is. If you expect the computer to read your mind, results will be the same as expecting a human employee to read your mind. LLMs are not Google or an encyclopedia, there are rarely "correct" or even "best" results in the traditional sense. Whether it's "worth it" depends on your use-case (hobby vs profit) and existing skill level with programming. If you're not already a programmer, you should start small on simple projects and learn best practices over time. Ask the LLM itself for recommendations on how your projects should be structured, ask it to build an architecture document to describe how it should work, and then ensure it adheres to it as you carefully build and test each function.

If you want my advice on what to do with your money, I would suggest signing up for a monthly subscription, seeing if/how it limits you, and then deciding based on that if it makes sense to buy local hardware. At the rate I was burning through Claude and Gemini tokens I could only work for an hour or two a day, and even then I had to avoid experimentation because I had to always worry about limits. This is the biggest advantage you get with local hardware: you've paid upfront so you're incentivized to use it rather than avoid using it.

I seriously doubt local LLMs will ever fully replace the utility of online LLMs, but local LLMs can certainly reduce your need for them to an extent that you could save a lot of money in the long run.

I started making projects with local LLMs about 6 months ago and I'm certainly nothing like a real "coder" at this point, but I doubt I would be in a better place today had I spent the same amount of time and money on formal education. My class would probably still be covering things I already knew from my very limited existing coding knowledge. From what I understand, after a year in a CS course your culminating project might something as simple as a Snake game or Tetris clone, etc. But on the other hand, I'm certainly learning things in the wrong order and leaving big gaps in my overall understanding. I think I'm much more inclined towards being a project manager than a coder though - I doubt I could possibly memorize all the various syntax that's required, for starters - and that sort of manager role is what you fit into when using LLMs.

Regarding 3090s, I would never suggest that. It is a newbie trap and such an obvious one it makes me consider conspiracy theories like some sort of vast international coalition of COVID-era 3090 buyers trying to break even. When you realize how limiting 24GB is (you will on the very first day) you'll immediately be throwing good money after bad. One 3090 can barely run anything of value, but what if I buy another one? Oh, but now I'm on the verge of being able to do this other thing... hmm... I do have a third slot on my motherboard... And then it's "is my output bad because I'm using a version of this model that's too quantized, or is it just not a good model? Maybe I'll try a bigger model with more quantized kv-cache? Maybe the opposite?" ... then you think, "I could sell these and get a couple 5090s", etc.

You have three real immediate options:

• Try an online subscription to at least see if they might be good enough for your usage pattern.

• Online hardware rental (ie runpod, etc) to preview how "local" models perform before spending big on your own hardware

• Purchase a real professional card tailored to local AI

If I was using online LLMs for my projects I would be spending over $1000 a month on API fees, easily. On some occasions I'd probably be spending that much in as little as a week. A larger upfront investment quickly pays for itself but only if your usage is routinely very high. But if that is the case, you should be looking at an RTX 5000 at minimum, ideally an RTX 6000 (both Blackwell generation, not Ada, etc). These are very expensive but they are, in my opinion, the real entry-level.

Another option is a pair of DGX Sparks (or their third-party variants) - these are the cheapest way to get 256GB of unified RAM for AI but they're hard to recommend without knowing your skill level with Linux and they'll only be about half as fast as 3090 in raw performance. Claude (etc) can actually guide you through the initial setup very effectively, but don't hear "256GB" and assume you can run a model of that size with respectable performance - an RTX 5000 Pro, despite its much smaller 48GB of VRAM, will run over twice as fast. And a 96GB RTX 6000 Pro will run more like 3-4x as fast.

That said, there are constantly things just over the horizon which might radically improve their token generation performance, and DGX Spark's prefill performance is actually quite decent. The overall performance picture might be very different in as little as a month or two - though these improvements will also apply to all other hardware.

I would not trade my single RTX 6000 with its 96GB of VRAM for six RTX 5090s with 192GB of VRAM. The thought of selling my DGX Sparks (Asus GX10 variant) however is constantly in the back of my mind, because despite their greater theoretical capability, the ease of use with my 6000 is so much greater.

If you want some hard numbers:
- 1 DGX Spark can run Qwen3.6-35B @ 4-bit quantization at up to 90 tokens per second
- 1 RTX 6000 Pro can run the same model @ F16 (no quantization) at up to 185 tokens per second
- 1 DGX Spark can run Qwen3.6-27B @ FP8 (its best, fastest quantization) at up to 18 tokens per second
- 1 RTX 6000 Pro can run Qwen3.6-27B @ 6-bit quantization at, IIRC, around 80 tokens per second
- 2 DGX Sparks can run Qwen3.5-122B @ 4-bit quantization at up to 40 tokens per second
- 1 RTX 6000 Pro can run Qwen3.5-122B @ 6-bit quantization at up to 120 tokens per second
- 2 DGX SParks can run Qwen3-Coder-Next @ FP8 at up to 60 tokens per second
- 1 RTX 6000 Pro can run Qwen3-Coder-Next @ 6-bit quantization at up to 145 tokens per second

The numbers I quoted for 6000 Pro with 6-bit quantization might've been 5-bit quantization, I don't remember for sure. I go back and forth on those, sometimes 5-bit is drastically faster, sometimes they're about the same. In rare cases such as Qwen3.6-35B, F16 is actually faster than Q8.