3090 48GB
Posted by tronathan@reddit | LocalLLaMA | View on Reddit | 65 comments
I was reading on another subreddit about a gent (presumably) who added another 8GB chip to his EVGA 3070, to bring it up to 16GB VRAM. In the comments, people were discussing the viability of doing this with other cards, like 3090, 3090Ti, 4090. Apparently only the 3090 could possibly have this technique applied because it is using 1GB chips, and 2GB chips are available. (Please correct me if I'm getting any of these details wrong, it is quite possible that I am mixing up some facts). Anyhoo, despite being hella dangerous and a total pain in the ass, it does sound somewhere between plausible and feasible to upgrade a 3090 FE to 48GB VRAM! (Thought I'm not sure about the economic feasibiliy.)
I haven't heard of anyone actually making this mod, but I thought it was worth mentioning here for anyone who has a hotplate, an adventurous spirit, and a steady hand.
Taiz2000@reddit
The short answer is no, it does not work. I have attempted this mod. While all 24x 16Gbit G6X modules work, the vbios can only recognise 24GB of vram. You need to mod the vbios to add a hypothetical "16x 32Gbit" entry for it to recognize all 48GB, for reference, the max support config in the vbios is 16x 16Gbit, which is what the 3090 is already using.
wikodeko@reddit
Have you tried this? https://www.techpowerup.com/vgabios/267498/267498 It seems like someone uploaded a vbios that supports 48 gigs
ar405@reddit
I've tried that after upgrading bios flash to 2gb and the gddr6x modules to double the capacity. Didn't boot. Reverted back to the original 1gb bios flash and booted with the default RTX 3090 bios just fine, but as you mentioned before it only sees 24gb.
48gb bios states support only for the gddr6 vram modules and not the gddr6x. That might be why. So waiting for the 48gb bios version with gddr6x support(
Duckers_McQuack@reddit
1 year later, have you found a bios that supports GDDR6x? As i'm slowly acquiring tools and hardware to perform the surgery, just lacking heat plate and hot air gun left, as well as finding memory modules i can buy.
Luans_Restorations@reddit
Have you gone through this? Are you trying with GDDR6x then?
yachty66@reddit
Why don't you just use GDDR6 2GB modules instead of GDDR6X?
CoderStone@reddit
Necro, but did you try soldering GDDR6 modules instead?
ar405@reddit
Nah, too much hassle already.
ThatDeveloper12@reddit
I suppose you could try GDDR6? (though it would require removing and resoldering new VRAM again)
ConteXCrown@reddit
i wonder if you could get 24gb with 12 chips to work.
Low_Finger4062@reddit
The mentioned Bios is for GDDR6, because it’s originally from the A6000. The A6000 is just a 3090 with GDDR6 and comes with 48GB. Using GDDR6X won’t work!
From the infobox: "Memory Support GDDR6, Hynix GDDR6, Micron GDDR6, Samsung GDDR6, Unknown"
wikodeko@reddit
Wow, thanks for sharing this information
ar405@reddit
It is twice as large as the original so you would need to replace flash memory module with a 2GB one before updating the bios.
yachty66@reddit
So, did you use the BIOS that is available as a link on that website or a different BIOS?
wikodeko@reddit
Did intend to reply to my comment?! You seem to have not read the whole comment thread.
Kiburici@reddit
The memory configuration is controlled by the straps on the board not by the vbios, this is how the memory is detected
Countertop_strike@reddit
Awesome, cool you tried it! Can you share more info about which card you modded (FE/Asus/EVGA?), which chip you used (Samsung/Micron?) and your process?
Also, did the card still work afterwards? Like it has 48gb of vram but still worked as if it had 24gb? I'm interested in giving this a go and it would be cool to know that if I go through all that work the worst that will happen is my card just works like it did before..
Taiz2000@reddit
Gigabyte Gaming OC, Micron D8BZC (iirc), unsolder old modules, solder new modules, modify straps according to board diagram Works but only 24G detected/available
TopMathematician5887@reddit
Can you cross reference a bios from RTX A6000 48GB with RTX3090 they are very similar in specs.
PraxisOG@reddit
Fuses are common in the silicon design of modern processors, and a certain combination of blow fuses on the gpu die tells the vbios "I'm a 3090". It is theoretically possible to mod the vbios for a 3090 to support more memory, which is how people are doing 22gb rtx 2080ti's, but no one has hacked the vbios to do that yet.
BlitheringRadiance@reddit
https://www.techpowerup.com/vgabios/267498/267498
Feeling-Bottle-8081@reddit
Does this work?
DNosnibor@reddit
It only supports GDDR6 not GDDR6X. But other than that, maybe.
miscab@reddit
Are you trying the 3090 48GB mod? I have 2,000 pcs of 3090 to have the memory bumped up to accommodate the LLaMA better.
phlurker@reddit
May I buy 2pcs?
az226@reddit
You have 2k 3090 GPUs?
miscab@reddit
Yes, I have. they are at rent now. The lifecycle will be greatly extended if memory can be doubled.
drumstyx@reddit
Did you end up finding something for this? From what I've gathered, 3090 might work, but with GDDR6 chips instead of GDDR6X and an A6000 vBIOS flash. Depending on the use case, the speed cut might be worth it.
Apparently 4090 can be doubled, but it's the 4090D that has it, since the folks doing it are in china, and of course 4090s can't go to China.
xrailgun@reddit
Did you end up trying the modded vbios others have shared?
Acrobatic_Land_5225@reddit
have you tried flashing Quadro Bios on RTX?
https://github.com/notfromstatefarm/nvflashk
https://www.techpowerup.com/vgabios/267498/267498
fallingdowndizzyvr@reddit
Doing stuff like this isn't new. People have done it forever. Piggybacking RAM is how I turned my 128KB Mac into a 512KB Mac.
If you are in China, doing stuff like this is easy. You wouldn't do it yourself but pay someone not much money to do it for you. Go to any Tech Center in China and you'll find plenty of people set up in their cubicles with the skills and equipment to do this. Just bring them the parts and they'll take care of the rest.
Speaking of China.
https://www.techpowerup.com/img/erPhoONBSBprjXvM.jpg
Current-Direction-97@reddit
Why are these kind of stalls not as common in Western countries?
fallingdowndizzyvr@reddit
Because we don't have the same tech culture here. China is all about tech. Go to Shenzhen and even the biggest gearhead in the West is just a bit player. The streets in the electronics district are literally littered in tech.
It's not just in Shenzhen. Pretty much every city in China has a big tech center. Either it'll be one high rise in smaller cities to multiple high rises in bigger cities. Each floor is easily the size of what a Fry's was or a Microcenter is. But much more dense. Nothing like the big wide open aisles those stores have/had. Anyone even remotely interested in tech should make a pilgrimage to a Chinese tech center at least once. It'll make your head swim.
connorharding098@reddit
Hey buddy, any update, or has NVDA wiped you off the face of the earth for discovering a loophole?
fallingdowndizzyvr@reddit
I literally have no idea what you are asking.
InevitableLight6@reddit
You were talking about acquiring 48 GB 3090s. Did they work?
dan-jan@reddit
You’re right - “48gb VRAM” GPUs are available, though I would say Taiwan electronics markets are a better source, followed by Shenzhen.
I’ll be getting my hands on a couple of 48gb VRAM 3090s, will update here soon.
Aware-Evidence-5170@reddit
Legend!
Good luck, hope it works.
dan-jan@reddit
This is my current build: 4090s but will probably plug the bootleg 3090s to see how it go
https://reddit.com/r/LocalLLaMA/s/yw1sPyZKzv
CoderStone@reddit
Can;t just dip like that man!
PoweredByMeanBean@reddit
Did it work?
gtek_engineer66@reddit
Did you do it?
ConteXCrown@reddit
any update?
BlitheringRadiance@reddit
Hi dan-jan - did you ever get your hands on some 3090s with 48GB VRAM?
Aphid_red@reddit
For what it's worth, I just apparently found these offered online:
https://www.c2-computer.com/products/new-parallel-nvidia-rtx-4090d-48gb-gddr6-256-bit-gpu-blower-edition?srsltid=AfmBOopCOTXkMhfmeBujr5_fXjMZYWfs_oW-yohW1diHonRs8GtC_io1
Not sure how trustworthy that is; 256-bit memory interface (a power of 2) doesn't make logical sense with the chip having a non-power of two amount of memory; that should be 2-sided 384-bit; 3GB / 1.5GB gddr6x chips don't exist yet in any nvidia gpu, they're always 1 or 2 GB. 48GB / 2GB * 64bit channel = 768 bit = 2 * 384 bit interface.
tronathan@reddit (OP)
Thanks for the reply! Yeah, i'm not *quite* curious enough to find out...
tripmine@reddit
I think it's likely this could work on a 3090, but probably not on a 4090. The 3090 uses 24 1GB chips and the 4090 has 12X 2GB chips. They don't make a 4GB chip unfortunately.
az226@reddit
Samsung made one, but they haven’t released it. But it’s also not G6X, just G6. Also if 16Gb G6X modules from Micron with the same clock and speed don’t work, then surely the 32Gb Samsung ones won’t, though it’s conceivably possible that the 16Gb ones do.
0xd00d@reddit
You just got me mentally salivating over 24x 4GB modules for 96GB of vram on a 3090. alas.
az226@reddit
192GB with NVBridge drool
1,920GB with p2p open kernel
0xd00d@reddit
whaaaat! p2p open kernel... this is geohot's doing? dear lord. Wait so are you saying 10 4090's can be.. wait seems 4090 would need nvlink to support it. 3090 has nvlink though. Why 1920GB? is 10 some kind of limit? Is this it? https://www.reddit.com/r/LocalLLaMA/comments/1c4gakl/got_p2p_working_with_4x_3090s/ Damn this is fun. Does this mean I should get 2 more 3090s? lmao, but how would I physically topologically connect them? two nvlink pairs?
az226@reddit
Not all 3090s can work via P2P. You don’t use NvLink when doing this. There is a server that has 20x SlimSAS x8 PCIe Gen4 slots.
So if you get 20 modded 3090s with 96GB each, that’s 1,920GB.
Cold-Diver-6354@reddit
Which server? This is really interesting
Mr_Moonsilver@reddit
Since they did it for the 4090, is there any update for the 3090? Could the vbios of the 4090 be used here?
0xd00d@reddit
they.. who did what for the 4090? 48GB???
Mr_Moonsilver@reddit
Yep, the 4090D. You can actually find a lot of 'gutted' 4090 pcbs on ebay right now, without the core
Schmandli@reddit
Does someone know how the speed of an inference scale when the Ram of a gpu is modified? Will it always be constant or is there a maximum capacity the gpu could handle? I don’t mean the bios or anything but just the logic behind it. Like how big can a matrixmultiplication get before the processor of the GPU is the problem and not the RAM of it.
Freonr2@reddit
Well, the short version is the model either fits into VRAM or it doesn't.
Schmandli@reddit
But I specificly asked for cases when the processor of the GPU is the bottleneck and not the VRAM.
ConteXCrown@reddit
if you have infinite vram the next thing be to bottlenecking would the memory bus be, because it can only put x much into to vram at a time
spyrosec@reddit
I've managed to find the 48GB BIOS, here:
https://www.techpowerup.com/vgabios/267498/267498
But I have not tested it. Any feedback is more than welcome
tronathan@reddit (OP)
Omg, I started reading this thread after getting a phone notification and thought, “wow, this guy writes a lot like me, that sounds exactly like something I’d say” - then I realized - I’m the OP.
Thanks, this is an interesting prospect.
If it was one or two chips, I might attempt, but if we’re literally talking about changing out all 24 chips, that’s a different story.
Still curious if anyone has done this and if it’s even possible. I also recall someone saying it wasn’t possible because of bus width or something.
ConteXCrown@reddit
u should try gddr6 because the 48GB enterprise gpus also use gddr6 and not gddr6x
ar405@reddit
This bios is for gddr6 not gddr6x, so either changing vram modules to ggdr6 or hoping it works at half the throughput as it is.
ConteXCrown@reddit
u should try gddr6 because the 48GB enterprise gpus also use gddr6 and not gddr6x