Higher capacity regular DDR5 timeline? 64GBx2 96GBx2?
Posted by capybooya@reddit | LocalLLaMA | View on Reddit | 26 comments
I'm struggling with my Google skills on this one, I seem to remember reading in the last year or so that higher density DDR5 would arrive soon. And for those of us running these models on regular desktop PC's, we want the maximum memory capacity in 2 DDR5 sticks for the minimum hassle. Does anyone know if there are higher capacity sticks and kits on the horizon anytime soon? We have had the choice of 2x48GB (96GB) for a while, and I'd hope to see 2x64GB or 2x96GB be available soon.
ps5cfw@reddit
What for though? Unless you are at least quad Channel you don't want to be using CPU Inference in big enough models (70b Is a shopping 1 t/s!) and even then not sure any model Is going to run fast enough at all!
RekoULt@reddit
Honestly,why do you want to be stagnant? improving pc tech is needed
ps5cfw@reddit
How Is making a dumb PC build gonna help with that, exactly?
CPU Inference Is Just not fast right now. That night change in the future, but that future Is not today, nor tomorrow.
RekoULt@reddit
If no improve or find or improve current tech,then we won't get it
I feel like am6/lga1900 or after might support it
I am noticing a Trend herr lol
Usb c mobo = no we don't want it
Ram speed = understandable as it doesn't effect anyone at this point
Ram capacity = reluctant as cpu needs support
Cpu speed = again same like ram
SSD = needed but people prefer SSD+hdd
Hdd = needed for everyone but only storage as everyone no longer use it for os (anyone who does is intentionally sabotaging himself)
Gpu speed and vram = needed ,no disagreement
Psu watts = waste for normal user,but needed for home servers
Monitor = unknown
Others are already peak so no more development on that like mouse, keyboard,pc case
Feel free to correct me,I welcome it
AutomaticDriver5882@reddit
This motherboard has 8 channels ASUS Pro WS WRX90E-SAGE SE
henfiber@reddit
This Threadripper motherboard only supports RDIMMs, though, and these are already available at 64+ GB per module sizes. Epyc motherboards are even better with 12 or 24 (for dual CPU).
AutomaticDriver5882@reddit
I like this MB also because I can put 4 x 4090s liquid cooled GPUs do those support that?
AutomaticDriver5882@reddit
Model number?
Expensive-Paint-9490@reddit
Epyc Genoa and higher have 12 channels.
AuggieKC@reddit
The one you posted?!
yeawhatever@reddit
I hear that often, and I can see why for chatting, but I use large models slowly all the time for refactoring code. Even at 0.1 t/s its much better than reprompting a small model 20 times.
jrherita@reddit
Some of the newer/better AM5 boards appear to be supporting 4 dimms at 6000+ speeds stably:
https://www.overclock.net/threads/asus-rog-crosshair-x670e-overclocking-discussion-thread.1799959/page-497
X670E ProArt Creator (ASUS)
Also Derbauer has a video on the X870E Crosshair Hero (I think that's the model) that also seems to support 6000+ speeds for 4 dimms.
I'd recommend more reserach before buying one of these boards, but it looks like 192GB @ 6000 DDR5 speeds may be possible now on AM5. (with 6000 being the 'sweet spot' for AM5 CPUs).
gfy_expert@reddit
If you can wait then wait. I’d rather think about ecc ram into consumer zone. I also always take 2 modules than 4 and have “beyond pristine” stable ram via karhu or gsat linux livecd but hey that’s me.
jrherita@reddit
According to the Samsung roadmap the dies should be in production by the end of this year, which presumes early next year for availability. https://www.forbes.com/sites/tomcoughlin/2022/10/06/samsung-reveals-2030-memory-plans/
However, another option in a month might be CUDIMM DDR5 DIMMs which *should* allow DDR5-6400 type speeds even with 4 dimms installed on a standard motherboard/CPU combo. CUDIMMs add a clock generator and will support 9000+ speeds reliably in dual channel mode. Zen 4/5 supposedly already can work with these types, and Arrow Lake is supposed to support it. tl;dr CUDIMMs should allow standard desktops to run 4 x 48GB (192GB) at 6000+ speeds in the next 2 months.
https://www.anandtech.com/show/21455/making-desktop-ddr5-even-faster-cudimms-debut-at-computex
JohnnyDaMitch@reddit
Seems exciting for Arrow Lake, but isn't it just bypass mode on Zen? I read the anandtech you linked, but I struggle to see how a CUDIMM that's bypassing the clock driver is any faster than current DDR5.
jrherita@reddit
Here's a newer article on the topic: https://videocardz.com/newz/msi-confirms-cu-dimm-ddr5-memory-support-for-ryzen-9000-and-8000-series
The article isn't completely clear that Zen will support the higher speeds (though the Infinity Fabric will still limit useful speeds to 6000-6400, faster doesn't gain anything performance wise). It says Zen 5 and Zen 4 APUs (9000 and 8000) series will work on MSI motherboards. Zen 4 support (7000) "maybe coming later".
Yes - I think density improvements is what Samsung has on deck. Though with CUDIMMs you might be able to do 4 x 64GB DIMMs at a reasonable 6400 speed or something like that.
We should at least see if 4 CUDIMMs can operate at a decent speed in the next 30 days. Arrow Lake launches 10/24, and I suspect some people will be testing CUDIMMs not long after. If Arrow Lake can handle 4 CUDIMMs at high speed then it's likely AMD will (after updates) do the same.
JohnnyDaMitch@reddit
I'd suspect there must be some unused socket PINs in DDR5 that are now for CUDIMM to communicate synchronization info for that duty cycle correction that was mentioned in the original article. I could be wrong though.
Downtown-Case-1755@reddit
TBH what you should hold out for is Strix Halo boards with LPDDR5X LPCAMM modules... that's the best hope.
Or just buy boards with 128GB/96GB usable soldered memory which is a more realistic hope.
Yes, it's "only" quad channel, but fast, and I suspect running vllm/exllama instead of CPU llama.cpp is going to make a huge difference.
JohnDuffy78@reddit
I saw the new X870E motherboards support 256GB with 4 slots.
I'm going with 2x48.
AnomalyNexus@reddit
Wouldn't 4x slots be faster for this, even if timings need to be looser?
Expensive-Paint-9490@reddit
No, it's two channels memory, so four sticks can be the same speed or lower, not higher.
Wrong-Historian@reddit
even 2x48 is still dual rank memory so still hard to run at high speed.
Wrong-Historian@reddit
2x48GB is pretty nice. Its slow but about the largest that you would want to run in CPU RAM. But even 2x48 is still dual rank memory so still hard to run at high speed. The ram I have is 2x48GB DDR5 6800, but with a 14900k it's still really hard to run it higher than 6400 and the timings/latency is pretty lousy as well.
first2wood@reddit
I copied and pasted your question and Perplexity tells me:
Expected Timeline for Higher Capacity DDR5
In summary, while you can currently find 2x48GB configurations, expect to see 2x64GB and potentially even larger configurations entering the market as early as late 2024, driven by new CPU architectures and increased manufacturing capabilities.
AutomaticDriver5882@reddit
ASUS Pro WS WRX90E-SAGE SE EEB Workstation Motherboard, AMD Ryzen™ Threadripper™ PRO 7000 WX-Series, ECC R-DIMM DDR5
I want this so bad
sammcj@reddit
I've got 4x 48gb @ 4600Mh and honestly - you never want your model to go into ram. It's useful for a ram disk and other things when moving and processing models though.