why GPUs have so little VRAM?

[-]

noiserr@reddit

Consumer GPUs are cost optimized around their intended primary use. So the manufacturers try to provide only what's needed. For gaming unlike professional workloads you don't need a lot. And if you need a lot more they are happy to up-sell you for their professional offerings. I really wish we get more VRAM on the next gen. Because AI really needs a lot of VRAM. Even 24GB is a pittance. Though not holding my breath.

Reply

[-]

mindcrimez@reddit

NVIDIA 6000 ADA is not a consumer card, which is limited to 48GB as op mentioned.

Reply

[-]

Darth_Caesium@reddit

I really wish the RTX 5000 series was like this: •RTX 5090Ti — 512-bit bus, 96GB of GDDR7 VRAM •RTX 5090 — 512-bit bus, 48GB of GDDR7 VRAM •RTX 5080Ti — 384-bit bus, 36GB of GDDR7 VRAM •RTX 5080 — 384-bit bus, 36GB of GDDR7 VRAM •RTX 5070Ti — 256-bit bus, 24GB of GDDR7 VRAM •RTX 5070 — 256-bit bus, 24GB of GGDR7 VRAM •RTX 5060Ti — 192-bit bus, 18GB of GDDR7 VRAM •RTX 5060 — 192-bit bus, 18GB of GDDR7 VRAM •RTX 5050 — 128-bit bus, 12GB of GDDR7 VRAM Notice the VRAM amounts. GDDR7 is going to have 3GB chips, so this could be done across the board like this to save on bus width. 5090Ti could be sold as an extremely expensive alternative AI card to "pro-sumers".

Reply

[-]

Mllns@reddit

5060 will have 12GB of GDDR6X if we're lucky

Reply

[-]

1soooo@reddit

12gb on a 96 bit bus that is

Reply

[-]

ET3D@reddit

I think that you're confusing things by comparing shipping VRAM to motherboard *capacity*. PCs don't normally ship with 128GB or 256GB, since very few people have need for this. They have the option for this, because RAM on motherboards is socketed. RAM on GPUs isn't socketed, and it isn't practical for it to be, so shipping a card with a large amount of RAM puts the burden on the manufacturer instead of the consumer, which makes it impractical.

Reply

[-]

Affectionate-Memory4@reddit

You can have that much vram. It's called an H100 or an Mi300. They use HBM, which is stacked vertically to make really dense memory packages, amd they use an absolutely massive bus (multiple kilobits) to move data around. On the consumer side, GDDR is used. The chips have a finite capacity and connect to the GPU with their own 32-bit lanes. You can have 2 per lane in a clamshell configuration, but that's where it maxes out. This means that the maximum capacity of a GPU is constrained by 2 things, the bus width and the maximum density of the memory modules. The densest GDDR6X I can find from Micron is 2GB/chip, so in clamshell that gets you 4GB Vram for every 32 bus bits. That gives your 384-bit bus GPU a maximum of 48GB of vram. That is as much as you can physically connect to the GPU. Denser Vram or a wider bus will allow more. This would mean your common GPU bus widths would max out at 48, 40, 32, 24, 16, 12, and 8GB for 384, 320, 256, 192, 128, 96, and 64-bit busses.

Reply

[-]

dstanton@reddit

Bring back 512-bit bus cards and give them 32gb!

Reply

[-]

FireSilicon@reddit

5090 and new quadros will supposedly use 512bit bus exactly because of this limitation.

Reply

[-]

Few-Age7354@reddit

Also Its not limition 24gb vram. Console use maximum 10gb vram, and all games based of them.

Reply

[-]

Dealric@reddit

They really are not 4k. Its heavily upscaled to 4k in 30 fps.

Reply

[-]

Few-Age7354@reddit

Not, it has only 10gb vram. PS5 is like rx 6700, not even xt module, and 6700 has only 10gb of vram and performs exactly like ps5. It hasn't in no way 12gb vram, lol! Only 13.9gb of Xbox all ram is for games(2.1 ram is for operating system), and you have to have at least 3.9 ram to play a game, so it leaves only 10gb of vram. In pc gaming, you have to have at least 16gb of ram, not including vram, so 3.9gb ram is even minimal assumption. In PS5 also some of the 16gb ram is for operating system, so it leaves same amount ram and vram(at least 3.9 ram and 10gb vram). Also expect that in PS5 pro, cause it will have the same ram as regular ps5. You have problem only with 8gb card in pc gaming, cause consoles use slightly more. Other card are fine. 12gb vram completely fine with 4k, it's more than ps5 vram. You can even play 8k max settings, without upscaling on 4070 super in horizon forbidden west and get 30fps, 8k! Not you regular 4k. 4080 that has 16gb vram can go beyond 8k regularly, 16gb vram for 4k is actually too much.

Reply

[-]

Dealric@reddit

Yes 16gb total for ps5. In general around 4k is used for other things and 12 for vram. 30fps is not good anyway. Also while 12gb is mostly fine. Just to remind some examples. Cp77 4k pt eats 17.

Reply

[-]

Strazdas1@reddit

PS5 devs i know said they target 8 GB of VRAM with other being for intense scenes or system usage. >Just to remind some examples. Cp77 4k pt eats 17. CP77 4k PT allocates 17 GB. How much it actually uses is a question for the ages as its really hard to test. youd have to artificially gimp memory in a way that it does not bottleneck elsewhere and test games performance with the limitation. We see that on 16 GB cards there is no issue with those settings, so it does not need 17 GB in actuality. Testing lower is harder. Esepcially since 12 GB cards offer inferior compute making tests harder to validate and artificially gimping memory is hard and creates its own validation issues (you cant just disable one chip and expect others to behave the same as before)

Reply

[-]

Dealric@reddit

Well how do you test it on 16 but not 12? You would need same card with different vram. 16gb cards are inferior to 20 or 24 gigs from same company

Reply

[-]

Strazdas1@reddit

>You would need same card with different vram. You now see why its hard to test actual usage instead of allocation.

Reply

[-]

Strazdas1@reddit

Consoles use a shared 16 GB with 6 GB reserved as syatem memory. PS5 dev targets 8 GB of VRAM on average.

Reply

[-]

Few-Age7354@reddit

Already disproven that 5090 will have any 512bit bus. Also it will have 24gb vram. Quadrus will have more. Also You don't need 512bit bus for more than 24gb vram. They can be 48gb vram with 384bit bus. 512bit bus means you have card with 600watts that will burn your case. 4090 already too massive and with big tdp. You can go beyond that.

Reply

[-]

kikimaru024@reddit

You can't prove a rumour until there's a release.

Reply

[-]

Cute-Pomegranate-966@reddit

It's not disproven, same person that said it won't have it said maybe it actually does.

Reply

[-]

chaddledee@reddit

Nvidia want people buying their enterprise GPUs for production workloads, especially AI stuff. I think they really regret giving the 3060 12GB of VRAM because it became the go-to card for people who want to dabble with AI workloads.

Reply

[-]

ZCEyPFOYr0MWyHDQJZO4@reddit

They don't regret it at all, otherwise they wouldn't have released the 4060 ti 16gb. The cores and memory bandwidth is too handicapped.

Reply

[-]

chaddledee@reddit

4060 16GB for $100 more than the 8GB, when the extra VRAM would cost them less than $25.

Reply

[-]

Strazdas1@reddit

They have sufficient RAM. why make the product more expensive to put in memory that 99% of your consumers will not use?

Reply

[-]

GenZia@reddit

The maximum capacity of a single GDDR6 IC is 16 Gb or 2GB, so that means you can have up to 24GB on a 384-bit wide bus (32-bit per channel). You can also have up to 48GB with clamshell configuration (à la Nvidia L20 and L40) but it won't give you more bandwidth + I don't think an 'average consumer' can fully saturate such an enormous vRAM buffer... at least in the year 2024. GDDR7 is going to change things slightly as it'll be available in 24Gb flavor, but that's only going to bump things up to 36GB (72GB with clamshell). If you want even more capacity, bandwidth or both, HBM is the way to go. I believe Nvidia Grace Hopper will be available with up to 512GB of HBM3e. Of course, HBM is an entirely different animal than 'traditional' GDDR ICs and is rarely seen in consumer (or even prosumer) space, mostly because of complexity and cost.

Reply

[-]

Netblock@reddit

>384-bit wide bus (32-bit per channel) Not that it changes anything, but GDDR6 does 16-bit-wide channels with 2 fully independent channels per BGA package. GDDR6 also includes an 8-bit-wide mode, which is how clamshell works like (they don't share the same pins like regular DDR, but instead split them up). As you can imagine there is a lot of CA-wire-count overhead, so GDDR6 also includes 'Pseudo-channel (PC) mode' which allows you to partially combine the CA to reduce the memory-controller-side pin count (data pins stays unique).

Reply

[-]

GenZia@reddit

I didn't realize that. I guess it depends on the memory controllers and whether or not they can handle all that bandwidth? However, GPUs 'traditionally' handle a single GDDR IC per pinout, each 32-bit wide. The only exception I can think of are old DDR2 and DDR3 GPUs which used 16-bit DDR ICs. For example, I used to have a GT440 with a 128-bit wide bus and DDR3 memory and it'd a total of 8 memory ICs on the PCB (2 x 16-bit per pinout).

Reply

[-]

Netblock@reddit

Well for the the '2x16 vs 1x32' thing, the bandwidth stays the same; the amount of data pins stays the same. The main difference is with regard to CA for command queue independency. Though, a consequence would be the the cacheline size the ASIC/SOC is built around. GDDR6 does a 16n burst length, which means the default configuration of GDDR6 of 16-bit-wide would land 32 Bytes of data per READ/WRITE. If I recall correctly, GPUs experience very contiguous access patterns so they would benefit from larger cachelines ([AMD does](https://www.amd.com/system/files/documents/rdna-whitepaper.pdf#%5B%7B%22num%22%3A28%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2C70%2C518%2C0%5D) 64Byte [cachelines](https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna3-shader-instruction-set-architecture-feb-2023_0.pdf#%5B%7B%22num%22%3A105%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2C0%2C595.86%2Cnull%5D); unsure about nvidia, probably the same). There are [also workloads that are so extremely random](https://www.servethehome.com/intel-shows-8-core-528-thread-processor-with-silicon-photonics/) that tiny cachelines would be preferable. It's extremely expected, but GDDR7 doubled the prefetch from 16n to 32n and doubled the channels from 2 to 4; keeping the 32Byte native cacheline.

Reply

[-]

vhailorx@reddit

Ram costs money. So less ram means lower prices and/or more profits for the oem. And there is a herding effect for game development that limits the utility of super large vram buffers (not many games spend the time to build for 24+ gb when that is less than 5% of the market. Additionally, nvidia in particular can charge 2-5x more for its professional products as compared to gaming gpus. But the silicon is effectively the same. So in order to minimize the self-cannibalization they use vram (and clock speed) to differentiate their geforce and qiadro lines. The former has higher clocks and smaller buffers; the latter has low clock speeds but massive buffers.

Reply

[-]

ttkciar@reddit

NVIDIA and AMD figure that almost nobody needs more VRAM, so they don't put more VRAM on the product, since more VRAM on the device costs money and their objective is to maximize profits.

Reply

[-]

godisbey@reddit

Only amd does that. Nvidia are still being shitty and not putting more that 8gb on their mid-range cards

Reply

[-]

nukleabomb@reddit

the 4060 and the 7600 are the same tier of card

Reply

[-]

Dealric@reddit

Not price wise

Reply

[-]

Nicholas-Steel@reddit

Graphics cards have for decades been designed around shuffling data in to and out of VRAM as needed, so a lot has never really been necessary. Now that General GPU Computing is finally starting to take off the required capacity of VRAM is rapidly rising as stuff like AI processing is a drastically different kind of work load to graphics rendering.

Reply

[-]

mrblaze1357@reddit

Idk I have 20Gb. Seems plenty to me, Nvidia is just cheap when it comes to vram. Gets you to upsell yourself to a more expensive card.

Reply

[-]

DJGloegg@reddit

My gpu has 20. And my previous has 11gb Thats plenty. Just dont buy nvidia cards

Reply

[-]

Just_Maintenance@reddit

Basically it's a different type of RAM, focused on bandwidth. CPU RAM usually focuses on power, or density, or latency. CPUs usually use DDR5, LPPDR5 or LPDDR5X. The individual modules max out at 32GB currently and have a maximum speed around 8.5GT/s GPUs usually use GDDR6 or GDDR6X. The individual modules max out at 2GB currently and have a maximum speed around 21GT/s.

Reply

[-]

Ghostsonplanets@reddit

Consumers don't need that much VRAM for gaming unless they're doing professional workloads (In which case, Nvidia will want to upsell you to a xx90 minimum or the Pro versions of their GPUs). Another factor is that VRAM is limited by bus size and GDDR capacity. Currently, Consumers GPUs top out at 384-bit and GDDR6 is manufactured at 16Gbit density (2GB). So, realistically, you can only get up 24GB of VRAM. Or 48GB with Clamshell. dGPU vendor could offer a GPU with higher VRAM capacity if they wanted, but the normal consumer would need to pay a much expensive price for it (Way higher than 4090 MSRP) and would also need to fight with professional workers who would be interested into a consumer GPU with so much VRAM and would easily be able to buy it at whatever price demanded.

Reply

[-]

skycake10@reddit

Pretty much everything you suggested as a factor at play here applies. RAM for GPUs is much faster, more power-hungry, and more expensive than main system RAM. It's also just a packaging issue. It's much harder to reach higher speeds with a DIMM slot, but you run out of room for traces to put that much memory on a single PCB.

Reply

[-]

AutoModerator@reddit

Hello! It looks like this might be a question or a request for help that violates [our rules](http://www.reddit.com/r/hardware/about/rules) on /r/hardware. If your post is about a computer build or tech support, please delete this post and resubmit it to /r/buildapc or /r/techsupport. If not please click report on this comment and the moderators will take a look. Thanks! *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/hardware) if you have any questions or concerns.*

Reply

Reply to Post

40 Comments