Why Do I Feel Poor Each Time I Decide to Buy a New GPU Even Though I Make More Money?

[-]

IrisColt@reddit

>And, it only comes with 32GB of VRAM. I'd still run the same models my now old 3090 can run... This.

Reply

[-]

Pretty happy here running Mistral Small 2503 Q8 at 16k context on my 5090. Don't get me wrong, I agree with the sentiment and have 3090s myself, but personally, 32 has been an appreciated increase from 24, and speed too.

Reply

[-]

CalvinsStuffedTiger@reddit

do you have 3090s + 5090s in the same machine? is there software that sends the compute to the appropriate machine based on the task?

Reply

[-]

Desm0nt@reddit

When smalls models like mistral 34b and qwq-32b became dirty cheap (0.1$ per 1M input, 0.3$ output) I stopped basically using them on my home 3090 because both context is smaller, quantized, and electricity comes out more expensive than API access. Basically, the home GPU is left for tasks that can't be done via API (Stable Diffusion/Flux training and use) or that are noticeably more expensive (long term extensive use of Vision models + it's finetuned variants). And for something more interesting (70b) we need again at least 2 3090s, and 5090 will not give much advantage here. It's easier to buy several cheap Mi60s with HMB2 VRAM and play directly with Deepseek R1.

Reply

[-]

MINIMAN10001@reddit

Also for me it's that by having a GPU you have access to all AI models. Video, 3D modeling, image Gen, voice cloning and anything else that decides to crop up.

Reply

[-]

Flat_Jelly_3581@reddit

Theres so many better ways to spend your money, just keep it, use it for a finetune, buy something cool with it, go someplace cool with it, invest it, buy another rtx 3090 for 48gb vram, etc.

Reply

[-]

Admirable-Star7088@reddit

As someone who uses AI mostly as a hobby, buying GPUs to run larger models is just ridiculously expensive. Unless I can use AI to earn money, it's just not worth that much money even if I can afford it, it feels like too much to spend on a hobby/toy. Instead, I upgraded to 64GB RAM and a more powerful CPU. I can not run larger models like 70b fast, but at least I can run them now, which is better than not being able to run them at all. Also, with new tech such as Speculative Decoding, 70b models on CPU is not that bad, I get around \~3-4 t/s with Llama 3.3 70b (Q5\_K\_M), which is quite impressive for CPU/RAM. Additionally, with a more optimized quant such as IQ4\_XS and Flash Attention enabled, you could probably get even more speed out of it (I still need to try this).

Reply

[-]

Iory1998@reddit (OP)

You should include KV Cache quantization too. I agree. That's why I am seriously contemplating buying and AMD Threadripper 5995WX. I found many good ones at around $1,000. I can still use it for 3D applications I am also fond of.

Reply

[-]

g33khub@reddit

Is 5995wx (and the whole platform around it) really worth it today? You get 8 channels but only ddr4 3200 whereas you can get ddr5 6400 on consumer for a lot lesser (if 192GB is enough). Then there is also strix halo 8000mhz 4 channel 128gb.

Reply

[-]

Iory1998@reddit (OP)

For me, it can be worth it since I am a 3D modeling hobbyist.

Reply

[-]

Creepy-Document4034@reddit

Where do you get a 5995wx for $1000?

Reply

[-]

skarrrrrrr@reddit

Is this activated per model or it's an Ollama setting ? I need to learn more about it

Reply

[-]

dr_manhattan_br@reddit

The problem today if compared to ten years ago is that ten years ago a GPU could be expensive. But the price tag was on the shelf for everyone to pay the same. It was a fair game where it was hard for everyone to buy one. Now days, you like a car, let's say a Mustang Dark Horse or a new Corvette or even the Porsche GT3 RS, and you look at the MSRP. You think: I can pay for it. But when you face the reality that a car that costs 70k, 120k or 140k in reality will cost 90k, 180k or 240k and this extra is lost money because after a few weeks or months, the price that will show up as an evaluation of your belong will be the MSRP minus depreciation. Where went all the extra costs that you paid? It went to someone with greed to charge you extra because the product is limited in quantity. You don't feel fair, like you felt ten years ago. And by the way, it is not fair. Still, you think: But I have the money, should I just enjoy the moment and pay the extra? This is the real question that you need to answer. I would say no, no matter how hard it is to lose the momentum, even if I have the extra money to pay. It is just not right. It feels like you own a restaurant in Chinatown and some guys come to your restaurant and ask for money protecting your place. Protect from whom? From themselves.

Reply

[-]

NectarineDifferent67@reddit

I think Mac might be a better choice if you want to go with a local AI model.

Reply

[-]

Iory1998@reddit (OP)

I am thinking about it but it just feels like I am paying way more that it deserve for the Apple logo.

Reply

[-]

HighWillord@reddit

You need to understand that if something is expensive, the bestbu can do its play with what u have. You don't need really to have 4k PC to be able to inference with AI. Just use what works the best for you. I would recommend it's trying to work with optimised models, because this feeling you have won't stop, and it isn't bad thar you want to buy that 4k GPU or whatever, the reality is with those items that they're expensive and sometimes they don't return the value of what you get. I recommend saving some money and trying to buy something new next year, I'm not an expert but i understand that if ur money needs effort and you want something that really adds value, then don't buy those things and wait until they're at offer, at some points they'll be.

Reply

[-]

SteveRD1@reddit

Agree. I finally stopped daydreaming about increasing VRAM and trying to figure out how to ever find cards so I'd be able to do fun AI stuff.. I now spend that time playing around with Deepseek Reasoner API and actually DOING fun stuff, not jut thinking about it. Vastly more performance at trivial cost.

Reply

[-]

DungeonMasterSupreme@reddit

It's not so much the value proposition to me as the opportunity cost. I'd *also* love to get a new 5090, and I also have way more money than I had growing up, but the prices get me thinking. Do I want inference to be a bit faster or do I want LASIK? Do I want to run slightly larger models, or should I take that really nice vacation to the south of Spain this summer? Is it really worth it to buy the GPU now? Couldn't I just put that money into stocks and come back in a year when maybe the supply's increased? Basically, the 5090 never wins when I pit its value up against the other things I could be doing with the money. If that's going to change one day, it's going to be because some massive breakthrough's been made with local AI, or the price will have had to drop a fair bit. Either way, one or the other is going to need to happen before I buy one.

Reply

[-]

Cannavor@reddit

5090 opens up a lot of PCVR modded games if you're into that.

Reply

[-]

simracerman@reddit

Since starting early January, I’ve learned so much about LLMs and tested models up to 32b parameters with my $500 Mini PC doing inference on a RX 680m iGPU with 16GB 48GB shared memory. It’s plenty fast for anything 14b and below. 24-32b take a few seconds to eval and output is usually 2.5-4.5 for this range of large models like Mistral 24b and QwQ 32b. I usually just ask my question and come back in 2-5 mins for the answer. Spending $2k before taxes will only give me speed over what I have, which I’ll admit is nice but not a ground breaking change since I’ll still be confined to the 32b models.

Reply

[-]

Iory1998@reddit (OP)

I am like you, and that's what frustrates me. I can't justify the price of the GPUs.

Reply

[-]

Cannavor@reddit

I've had some shitty laptops over the years because I needed a computer for basic tasks. Never could justify the price of a desktop plus all the peripherals. Now I have finally decided to build my first desktop pc and I have the money for a 5090, but I can't find the damn thing in stock. I'm not about to pay for one of the models with a $700 cooler that gets the same performance as the FE's cooler and I'm definitely not about to pay scalpers. If I already had a 3090 I'd just stick with that. Not worth the hassle of trying to actually but a 5090 for the bump in performance.

Reply

[-]

Herr_Drosselmeyer@reddit

Simple: your income + raises don't outpace inflation. But don't worry, the same is true for 99% of the population.

Reply

[-]

ogaat@reddit

You should budget the money in advance, adding in a generous allowance based on growth trends. If the card is STILL beyond that, then remember Groucho Marx, "No one goes to that club any more because it is always too crowded"

Reply

[-]

LostHisDog@reddit

There's a thing called the asshole tax where the highest end products provide the least bang for the buck. In the past, there weren't enough assholes to really impact product rollouts for large corporations but in the last few years the number of assholes willing to empty their wallet just to get bragging rights to a x090 series card has shot through the roof. Couple that with the fact that even non-assholes wanted to get rich off the blockchain or AI or whatever the empty promise of salvation was their siren call and anyone who just wants a gaming card for gaming is well and truly fucked. The only real good new out of all this is that since no gamers have been able to get any gaming cards the games all still mostly have to run on potatoes if the dev wants to sell them to anyone... enter the era of the Steam Deck. So... yeah... don't be an asshole and try not to worry about it. Grab a used something or go rogue and get one of those Intel cards and see if you can play anything on that.

Reply

[-]

rog-uk@reddit

What burns me is that before Xmas I got a 4080 Super and although it was the best one that would fit in my tower case, the 4090 was twice as expensive for only 50% more VRAM/Compute - so even if I was minded to pay for it, I would have felt ripped off - Nvidia obviously knows exactly what it's doing here. Gits.

Reply

[-]

Ninja_Weedle@reddit

You're...you're just on the unluckiest upgrade cadence imaginable. you're not losing it- 5090 prices are insane and nobody should be dropping 3k+ on a card that might just combust anyway. The market will get better, hold onto that 3090 for a few more months. don't buy from scalpers, that will only make it worse.

Reply

[-]

Prince_Noodletocks@reddit

I only ever felt like it was a proper upgrade when I went from a single 3090 to 2x3090s, then going to 3xA6000s from 3x3090s. I honestly never should have bothered with the third 3090 and just skipped to A6000s. I have a different rig with the 3x3090 setup now but it seems a waste.

Reply

[-]

Woodboah@reddit

Inflation

Reply

[-]

ParaboloidalCrest@reddit

You obviously come from a frugal background. Good for you. And yes, GPU prices are always insane due to the oligopoly of the supply.

Reply

[-]

martinerous@reddit

I had similar feelings in my recent thread here: [https://www.reddit.com/r/LocalLLaMA/comments/1j6vmke/is\_rtx\_3090\_still\_the\_only\_king\_of/](https://www.reddit.com/r/LocalLLaMA/comments/1j6vmke/is_rtx_3090_still_the_only_king_of/)

Reply

[-]

ttkciar@reddit

The market for new GPUs is kind of ridiculous. I wouldn't pay that kind of money for niche hardware which will lose half its value immediately and become obsolete in a few years. What I've done is limited my GPU purchases to eBay, for older tech which is *already* obsolete, and made do with less capable hardware (which has been fine, for my purposes). Refurb 8GB GPUs can be had for less than $100, and I picked up my AMD Instinct MI60 with 32GB of VRAM for $500 (and it jfw with llama.cpp/Vulkan on Linux). Living behind the curve is pretty comfortable living, and very affordable.

Reply

[-]

No_Afternoon_4260@reddit

From my 2 years experience these gpus tend to gain value not become obsolete immediately 😅

Reply

[-]

Such_Advantage_6949@reddit

3090 price is going up lately. Used 4090 is asking for almost as much as new when i bought 2 years ago or even more

Reply

[-]

No_Afternoon_4260@reddit

Better than fine wine 🤣

Reply

[-]

Liringlass@reddit

That’s the crazy thing now haha

Reply

[-]

cobbleplox@reddit

I don't even know if it's that much "living behind the curve". Even people buying the hottest new thing are not buying every new iteration (at least I hope so), so they are behind the curve most of the time too. I bet there are scenarios where you could actually be closer to the curve on average for less money by buying older/cheaper stuff more regularly.

Reply

[-]

pier4r@reddit

> Living behind the curve is pretty comfortable living, and very affordable. this should be a more common approach.

Reply

[-]

OriginalPlayerHater@reddit

havent you learned from Tate, its the matrix holding you down man! also i think gpus suck for llm usage right now, all i want is 12 more gigs of vram on my 6700xt i paid 250 for, i dont need a fuckin nuclear reactor and i dont want to pay 700 for a used 3090 for it. right now im using cloud providers until they come out with low tier, high vram hardware that's around 500 for 40 or so gigs of unified or vram. until it hits that point for me personally its not worth it

Reply

[-]

skarrrrrrr@reddit

Im just using what I have and use offloading where I can.

Reply

[-]

ThisGonBHard@reddit

Because GPU prices went crazy, outpacing inflation by far, and for AI, NVIDIA is also selling low end silicon as high end (5080 is in fact, not an x80 class card historically, it is not even a 70 really...). Right now, there are the second worst historical prices ever, behind the pandemic ones. If you want to get some GPUs for AI, get multiple 3060 12 GB. 4 of those are the cost of 1 5070 MSRP, and have 48 GB of VRAM. If not, rent it, it makes more economical sense sadly.

Reply

[-]

Massive-Question-550@reddit

It makes sense. Sometimes it doesn't matter how much money you make, if something seems to have really bad value you won't want to buy it. For example I'll never pay 10k for the best tv, it doesn't matter how much money I make, nothing will convince me that a TV is worth 10 grand as a cheaper tv does everything I feel a TV needs to do and the value tradeoff isn't worth it.

Reply

[-]

-p-e-w-@reddit

The funny thing is that many, many people will happily pay $3k+ for some designer jacket. Which is just a bunch of (usually cheap) fabrics sewn together. But of course, a supercomputer that fits in a shoebox, which you can use to create a digital companion or a pair programmer that does whatever you tell it to, cannot possibly be worth even half as much…

Reply

[-]

Liringlass@reddit

There is a difference between what it can be, and what it will actually be. It can be a pair programmer that will work for years with no salary, except if you don’t need code it doesn’t really achieve its potential value. And if you really need a pair programmer won’t you be better off using larger commercial models anyway, that will cost less in a few years than your dual 5090 investment, while offering better results and giving you the ability to switch provider when something better comes out? Local AI is great for many reasons, and it’s fun, but it’s either an expensive hobby, either a business decision based not only on the value it brings but also on other considerations such as privacy etc.

Reply

[-]

-p-e-w-@reddit

> And if you really need a pair programmer won’t you be better off using larger commercial models anyway, that will cost less in a few years than your dual 5090 investment, while offering better results and giving you the ability to switch provider when something better comes out? Sending your code to a cloud server is a non-starter in many environments.

Reply

[-]

Liringlass@reddit

True, in this case local AI is the way.

Reply

[-]

Iory1998@reddit (OP)

That's my position as well. It's a hobby. Trying new models and experience how this technology evolves first hand is exciting. I learn so much from playing around with local LLMs than GPT or Claude. It's an expensive hobby.

Reply

[-]

alamacra@reddit

It's not a supercomputer. A supercomputer is a high quantity multiprocessor system that acts far beyond what common consumer hardware can do, and a single GPU accelerator is not that. It can't even do anything by itself, since it's not a complete machine. El Capitan, an actual supercomputer, comprises 44k CPUs and 44k GPUs respectively. >you can use to create a digital companion or a pair programmer A rather crappy one, since you are limited to a measly 32GB of VRAM, when something like R1 needs at least 512GB to run at Q4 with at least some context. Even if you bought ten of these, it wouldn't be enough. Also, for inference at that amount of memory, you don't even need that, you can just use RAM, and will in fact end up better off just by using a 8+ channel DDR5 system.

Reply

[-]

socialjusticeinme@reddit

It’s a little different with clothing and jewelry - if you spend $4000 on a ring, 10+ years from now it’ll probably be worth $4000 or more. Let’s say you spend $4000 on that shiny new Mac Studio - it’s gunna be obsolete trash in 10 years and a collectors item at best.

Reply

[-]

-p-e-w-@reddit

Lol. You picked jewelry when I specifically mentioned clothes. What you’re saying is absolutely *not* true for clothes. A jacket that is all the rage in 2025 will be near-worthless in 2028. And 10 year old Macbooks aren’t collector’s items, what are you talking about? Many people use those as their daily driver.

Reply

[-]

Iory1998@reddit (OP)

People will always make some irrational decisions. We don't always buy things because we need them. Sometimes, we buy on impulse. But for me, splashing a month worth of salary on a GPU is a bit too much.

Reply

[-]

cobbleplox@reddit

> if you spend $4000 on a ring, 10+ years from now it’ll probably be worth $4000 or more Afaik this is very much not true and a really bad idea to justify buying jewelry this way. Because a lot of the price comes from other things than raw materials. And... probably doesn't make much sense to mention in this context, but diamond prices weren't very happy about industrial diamonds either.

Reply

[-]

HanzoMainKappa@reddit

inb4 jensen huang tom ford black leather jacket

Reply

[-]

Autobahn97@reddit

lol NVIDIA makes money faster than you do! I'm really hoping once both NVIDIA and AMD have their GPUs out there and there is no shortage that prices will normalize a bit but I think $750+ is the new norm for getting into a more powerful GPU at Radeon 9070 xt level or better.

Reply

[-]

Ska82@reddit

Same here. It is even worse for developing countries with depreciating local currencies and duties/ tariffs. when the 50x0 series were announced, i ended up buying 2 new 3090s when the price fell (before it became unavailable on amazon). sometimes u can still do amazing stuff with reasonably older nvidia cards

Reply

[-]

aadoop6@reddit

New 3090s? I thought they were non-existent.

Reply

[-]

Ska82@reddit

yeps. i think i ended up buying the last 2 3090s on amazon cos' i never saw them again

Reply

[-]

Blindax@reddit

The problem is that even if you consider the price of 3k, those are still a bargain in terms of of your ROI. Until there is real competition it will remain high. But I also feels I paid too much for mine with my own money 🥲

Reply

[-]

Iory1998@reddit (OP)

I feel ya. My last rig cost me $2500 in 2021 and I had an RTX 3090, back then! Remember due to lack of demand, RTX3090 was selling at around $2000-2500 and people were complaining. Never did I think that I would see a GPU at around $8000 with 32GB of VRAM! I'd rather buy RTX6000 Ada for that price.

Reply

[-]

Blindax@reddit

Well those who buy 5090 at that price are either stupid or don’t care about money.

Reply

[-]

Iory1998@reddit (OP)

They are called early adopters. They usually absorb most of the cost early on.

Reply

[-]

giblesnot@reddit

I make really good money, but I still only buy used GPU. I can buy an interesting used gpu every 18 months and still not spend that much. I bought a 12gb 3060 for under 300 on eBay when Llama first released/leaked. I bought a 3090 the year after for 700 when the prices settled down for a bit. I bought a dual xeon 128gb server (not a gpu, I know) with an m2000 in it for around 300 this year, added the 12gb 3060 and put a quadro m4000 (8gb, $64) into my desktop next to the 3090 to get a combined 32gb. Now I can run 32gb worth of llm on my desktop and a second smaller code model on the server, and I'm in less than $2000, (including a psu upgrade) in llm focused hardware in 2 years. I also taught myself proxmox, pci passthrough, and gpu containers while I was at it. The used market is definitely staying interesting, while the bleeding edge 5090 feels like a bad value. That said, eBay is full of crooks. Check those seller ratings and descriptions very carefully.

Reply

[-]

Iory1998@reddit (OP)

Yeah man, I am like you, I always make rational decisions when I buy. My motto is simple: If I don't use something more than once a week, I probably don't need it. But, you can already see where the LLM trajectory is heading! With reasoning models, speed becomes important because you can't just wait for a 32B model (yes, I am talking about you, QwQ-32B) thinking for 5 minutes to respond generating tons of heat and wasted energy.

Reply

[-]

giblesnot@reddit

I live somewhere that is very cold 7 or more months of the year so I figure all the heat output is just a fringe benefit :-D

Reply

[-]

Iory1998@reddit (OP)

Haha. Right, that's not wasted heat at all.

Reply

[-]

giblesnot@reddit

My next big project is going to be trying one of these frameworks that let's you spread lama.cpp over multiple servers to serve larger models.

Reply

[-]

DeltaSqueezer@reddit

I had the worst timing. Sold my GPUs just before the covid crisis waiting for new GPUs. I 'lost' $300 in value as I didn't realise what had happened to prices and was without GPU for a long time. In the end, I had to buy an APU to tide me over.

Reply

[-]

Iory1998@reddit (OP)

I had to relocate to China, and I gave my pc with the water cooled RTX3090 to him (he is a gamer). I could have just brought the GPU with me, but I decided against it. After all, I'll be going to the place where the GPUs are made, right? WRONG! That's a big miscalculation as the GPU prices here are more expensive than in my own country. The RTX3090 cost almost $900. But once thing I can tell you. The Chinese are exceptionally talented, I work with them on a daily basis. I have no doubt in my mind that in a few years they will produce their own GPUs, and that will bring the market back to a sane level.

Reply

[-]

DeltaSqueezer@reddit

I can only hope so. These sanctions against Chinese semiconductors are really hurting consumers. I also work with embedded chips and there was so much innovative and cheap ICs being designed in China, it would be a shame if these all got killed off. I suspect had China been left alone to do its own thing, we would have had some really interesting products over the coming years.

Reply

[-]

Iory1998@reddit (OP)

Do not believe what the Western media tells you about China. Life in China is really cozy, affordable, and peaceful. I can smell Chinese GPUs coming in the near future.

Reply

[-]

Turbulent_Pin7635@reddit

M3 Ultra 512GB

Reply

[-]

Iory1998@reddit (OP)

🤦‍♂️🤦‍♂️🤦‍♂️

Reply

[-]

Turbulent_Pin7635@reddit

I know I know

Reply

[-]

LocoLanguageModel@reddit

Say you are a developer and you have $500,000 in savings; dropping around $10,000 on some crazy high vram won't impact your day to day financial life, but it is still 1/50 of your worth when you can use Claude for equal to or better results. It's easier to justify $30,000 on that car you might need, because the alternatives are inconvenient and worse. Need vs want.

Reply

[-]

a_beautiful_rhind@reddit

It's not just GPUs. I feel like I'm never getting any more money, ever, when I eye a larger purchase. Maybe you got some trauma from being broke. I sure do.

Reply

[-]

Iory1998@reddit (OP)

I feel ya buddy I really do. It's not trauma, but managing expectation and showing discipline.

Reply

[-]

a_beautiful_rhind@reddit

That's what I tell myself too. Sometimes you gotta enjoy yourself tho.

Reply

[-]

sunshinecheung@reddit

You can buy nvidia stock in 2023, lol

Reply

[-]

Iory1998@reddit (OP)

I was in a different country back then and it was hard for me to send money outside due to fiscal restriction. I wanted to buy Nvidia stock when Stable Diffusion launched in August 2022. The week it launched, I installed it and tried it (SD1.4). I immediately knew NVidia stock will get valuable.

Reply

[-]

ThinkExtension2328@reddit

Price to performance babyeee , compare that new 1000$ gpu from 10 years ago. Say the pc you built to play GTA 5 in 2013. Performance is allot better nowadays despite what you hear. Also we have hit a wall with chip manufacturing you can only make components so small before they start having quantum errors (not a joke). Thus all the small increments now days with hardware. We probably need some new technology advancement.

Reply

[-]

Iory1998@reddit (OP)

Didn't you forget something important... Something that rhymes with corporate greed and monopolistic position?

Reply

[-]

Interesting8547@reddit

Though there is a lot of demand for Nvidia GPUs currently, so the prices are not normal. But there is one other thing, salaries don't keep up with inflation... so people actually get more poor with time. Some people would say "but we actually get rich with time"... no we don't, the base level goes up, because of technology and production, but the average person wealth goes down. And the rich people... they get more insanely rich than ever in history, but that wealth does not get redistributed downwards much. I'll just give one simple example, in the past it was more expensive to get a nice big fridge than a GPU, now with the price of 1 GPU, you can take a fridge and a washing machine.... but that does not mean "we've become wealthier".... just the base level has gone up, and having a "washing machine" is basically nothing. The thing is, Nvidia GPUs are currently expensive because there are not enough of them, and because we are more poor than before... i.e. if our wealth has really gone up with the real level of inflation (not the fake one), we would have been able to buy GPUs easily, even for a $1000.... but that's not the case, because average salaries are not catching up, not even close.

Reply

[-]

socialjusticeinme@reddit

Uhh, the 5090 is the very top of the line for consumer GPUs. Have you ever priced fridges? If you goto the top of the line brands you can drop $10,000 on a fridge. Even the regular brands like LG and Samsung have different model tiers and their highest tiers are most definitely more expensive then the 5090.

Reply

[-]

Iory1998@reddit (OP)

Fair point there. That's a great observation.

Reply

[-]

ThinkExtension2328@reddit

I mean yes that exists but falls more on the gpu ram amounts not so much the performance of the cards it self.

Reply

[-]

Massive-Question-550@reddit

True but the limit for consumer cards isn't processing power but ram speed and ram capacity, hence the demand for Frankensteined 48gb 4090's being so strong that they literally create new pcb's and modded software for them.

Reply

[-]

Iory1998@reddit (OP)

I live in China for now, and I can tell you those 4090 48GB are everywhere, and they are still expensive. I looked around and I think an 2xRTX A5000 might offer a better alternative.

Reply

[-]

Iory1998@reddit (OP)

Vram prices have dropped significantly. When you have one major player having monopoly, that's the unfortunate outcome.

Reply

[-]

Interesting8547@reddit

Even if there were more companies TSMC is only one, also there are probably other factors at play, which are invisible for us. The question is, why Intel or AMD does not do a GPU with a lot of VRAM (because VRAM is cheap) , but they are complacent with the current situation... isn't that strange?! Also market demand does not mean other players can just "pop up" out of nowhere. There are also other things at play... for example one of the things is, context understanding AI was mostly ignored by most people as a possibility and was considered "a pipe dream" until just a few years ago, so nobody exept Nvidia, invested in it... and even Nvidia was not a 100% in, but just barely. Probably now Nvidia is 80% in and is one of the companies most invested in AI currently.

Reply

[-]

Iory1998@reddit (OP)

What saved Nvidia is ray tracing since they use AI for denoising and path prediction. Nvidia wanted to create a simulation world where companies can design their plants and production lines, something that is super complex to achieve in real life.

Reply

[-]

ThinkExtension2328@reddit

The vram argument is the only area I agree with you this is monopolistic behaviour. However the rest of the gpu is hitting a wall it’s the best it could be.

Reply

[-]

mustafar0111@reddit

That is it in a nutshell. Nvidia can charge more due to lack of competition. If AMD had not stepped in with Ryzen we'd have the same thing still going on in the CPU space.

Reply

[-]

NNN_Throwaway2@reddit

Uh, no. Price to performance is probably the worst its ever been. This is both the fault of Nvidia for price gouging and the games industry for shipping games with worse and worse optimization.

Reply

[-]

Massive-Question-550@reddit

Also AMD, their only long standing competitor regularly offering only slightly better value but less overall performance and less features. That's finally changing a bit with the 9070xt but the fact is that an enthusiast typically wants the best gpu, so a 9070xt might not cut it.

Reply

[-]

NNN_Throwaway2@reddit

AMD being on the ropes and failing to compete since 2006 has allowed Nvidia to charge what they want and to not push as hard as they could on performance.

Reply

[-]

Iory1998@reddit (OP)

AMD who? That boat sailed years ago for me. It's not now AMD will compete when it had many years to do that. I feel like AMD's focus now is on the enterprise market for datacenters. Either way, consumers will get whatever Nvidia wants them to get. My bet is on Chinese tech companies who also feel the discrimination and are motivated to come up with alternatives.

Reply

[-]

ThinkExtension2328@reddit

Compared to what though? By your logic you should be able to buy a 6year old amd card and have it be better then a 5090. Again I don’t give a shit about team red or green here computing as a whole is hitting a wall.

Reply

[-]

AppearanceHeavy6724@reddit

9 years old 1080 is about same as 3060 and better than 4060 for LLMs what are you talking about?

Reply

[-]

ThinkExtension2328@reddit

We are talking as a whole and yes your correct which is what makes the idea of buying new gpu’s for llm’s stupid unless your buying for vram. We use a very small subset of a gpu’s capabilities. Which is where things like the A2000 make more sense as it only uses 75w.

Reply

[-]

NNN_Throwaway2@reddit

Compared to previous generations. Price to performance has been gradually declining over time. There's no indication that performance is anywhere near hitting a wall.

Reply

[-]

ThinkExtension2328@reddit

You realise the oxymoron in your own statement right?? Performance is degrading over time - that’s literally the indication and mores law at work.

Reply

[-]

NNN_Throwaway2@reddit

I didn't say performance had been degrading over time.

Reply

[-]

ThinkExtension2328@reddit

Sorry declining , verbal gymnastics or not. We need new generation of computer hardware. Hopefully we start seeing more use of galium based chips tho people already bitch how expensive silicon chips are.

Reply

[-]

Massive-Question-550@reddit

You are going to see hybrid light based transistors in GPU's before you see gallium I think. Either way I think we have about 10 years tops with the same transistor tech before we truly hit a wall.

Reply

[-]

ThinkExtension2328@reddit

Is that tech even in the testing phase tho? (Light based transistors?)

Reply

[-]

NNN_Throwaway2@reddit

I didn't say it was declining either. Read.

Reply

[-]

EmilPi@reddit

You are right, I don't understand why someone downvotes you. But of course there is hype, high demand, and NVidia's monopoly too.

Reply

[-]

xrvz@reddit

That doesn't even include the price of a new house once the 5090 burns your current one down.

Reply

[-]

Iory1998@reddit (OP)

Haha this comment is pure gold! 😂

Reply

[-]

abitrolly@reddit

Relax. Buy a sushi laptop and enjoy the World slowly, at 0.5 t/s

Reply

[-]

pier4r@reddit

it all depends, whatever the purchase (even for something that costs 10$) on the utility that you see in that purchase vs the utility that you see in the money spent somewhere else. IIRC that is "opportunity costs". Hence if the GPU is not really going to bring you the value that you expect for the cost, doesn't matter if you make more money.

Reply

[-]

Strange-History7511@reddit

Why not just make more money?

Reply

[-]

Iory1998@reddit (OP)

Didn't you read the post? That's the whole point of writing this post. The GPU prices don't stay fix in time. Each time I go make more money, they get more expensive.

Reply

[-]

Blindax@reddit

I thought initially that you were making money about the GPU. But at the end it’s a bit the point, every feels the same but those who make money out of the GPU and this is likely a good part. That is why they are so expensive and until there is competition to run AI with a good performance I don’t see the price going down soon.

Reply

[-]

nas2k21@reddit

This time, go make more, more money, gotta get a step ahead of the game 😎

Reply

[-]

Kornelius20@reddit

What if OP earning money is what's causing GPU prices to rise?

Reply

[-]

nas2k21@reddit

Hadn't considered that, so you're saying if we crippled op's income gpus go down?

Reply

[-]

Strange-History7511@reddit

Bingo.

Reply

[-]

Klutzy_Comfort_4443@reddit

Then, make even more money.

Reply

[-]

KillerQF@reddit

Nvidia + TSMC lack of competition

Reply

[-]

Sambojin1@reddit

Buy a ~$1000 dollar Snapdragon 8 Elite phone with 24gig of ram, run slightly smaller ARM optimized LLMs *at the same time as doing other stuff"*, thus increasing throughput by whatever the phone gives you, and walk away from the new-tech problem for another year or so. Or just use the stuff you already have, and spend $0. Both are good options.

Reply

[-]

mustafar0111@reddit

What I did was pickup two P100's for $240 USD for now. Using that plus my desktop with an RX 6800 until Strix Halo gets released and reviewed. If its as good as it seems I'll pick up one of those with a max RAM loadout.

Reply

Reply to Post

122 Comments