SK hynix starts mass production of 192GB SOCAMM2 for NVIDIA AI servers

Posted by OkReport5065@reddit | LocalLLaMA | View on Reddit | 48 comments

hynix just started mass producing a 192GB SOCAMM2 memory module aimed at next gen AI servers, and it is basically trying to fix one of the biggest bottlenecks in modern AI systems. Instead of traditional server RAM, it uses LPDDR5X like you would find in phones, which lets it push more than double the bandwidth while cutting power use by over 75 percent compared to RDIMM. It is also being built specifically for NVIDIA’s upcoming Vera Rubin platform, which tells you this is all about feeding massive AI training workloads. GPUs get all the attention, but memory is quickly becoming the real limiter, and this feels like a pretty clear shift in where the industry is headed.

[-]

Fluffywings@reddit

Gpus with customizable VRAM is a potential near future (3 years) based on leaked documents. This would allow people to really scale their systems to their use case.

[-]

FreezeS@reddit

What do you meam by "people"?

[-]

somerandomperson313@reddit

Microsoft, OpenAI, ect.

[-]

tat_tvam_asshole@reddit

"corporations are people, my friend"

[-]

bick_nyers@reddit

Sorry, but I seriously doubt that.

[-]

Fluffywings@reddit

I don't think it will happen for day to day people. It will start at workstation levels using LPDDR6x. AMD has LPDDRX for the 10th or 11th series just not CAMM2.

[-]

Confident_Ideal_5385@reddit

Wonder if it's gonna turn out that Hynix bet on the wrong horse here.

The ass will fall out of the neodatacentre market eventually, the only question is how long the balloon stays in the air.

[-]

Piyh@reddit

The richest men in the world in charge of some of the most profitable companies in human history are all chasing the same goal. They are getting the chance to build new companies larger than the ones that established them the richest men in the world.

The tokens they sell are incredibly valuable, have a nearly infinite TAM, and are the stuff of science fiction.

The bottom is not going to fall out.

[-]

Confident_Ideal_5385@reddit

They're valuable, those tokens, but they're not cost-recoverable. Which is fine for Google, MS, Facebook, where the clanker is a loss leader or whatever. If your core business model revolves around monetisation of tokens, you're gonna have a hard time pricing those tokens in a profitable fashion.

[-]

No-Refrigerator-1672@reddit

They have hard time pricing those tokens today. This does not matter at all, as the moneybags behind them are fine of burning billions yearly, if that means having the market leadership 5-10 years later. Every investor is playing long-term game. Besides, a ~30B model is absurdly cheap to host, and we can see that today's 30B match in quality yeasteryear's SOTAs. This means that hardware and electricity prices of token will go down due to more efficient models, so the companies have clear path to future profitability, but the market share needs to be conquered today. Therefore, don't count on big providers bursting in rhe next few years, that's not going to happen.

[-]

Confident_Ideal_5385@reddit

Alphabet, MS, Facebook are playing a long long long game and can afford to burn through hundreds of gigabucks. I doubt Anthropic can, and I'm sceptical of how much longer OpenAI can keep the burn rate up.

30B dense models from this generation are fucking amazing, but there's no SaaS moat there. People will bring em inhouse for sovereignty reasons if nothing else.

The "market share first" mentality works if you're trying to create a network effect like FB or whatever. It's completely useless with something like a clanker. Raise the prices and see how little stickiness your offering actually has.

[-]

No-Refrigerator-1672@reddit

I'm sceptical of how much longer OpenAI can keep the burn rate up.

A quarter of OpenAI is owned by Microsoft. If Microsoft can burn money, so does OpenAI. Both Google and Amazon are investing into Anthropic, they have the money to burn too. If any of them run out of money, their investors can always give more.

30B dense models from this generation are fucking amazing, but there's no SaaS moat there.

You completely missed my point. The fact how quick 30B category develops means that in just a year or two AI SOTA companies can have in-house small models serving clients at very low serving costs. They totally have ways to bring down the costs, it's not the priority right now.

The "market share first" mentality works if you're trying to create a network effect like FB or whatever. It's completely useless with something like a clanker.

Hello? Are you living under a rock? Securing the world's main AI provider position is the path of having immense profits, as AI will be used across all industries and all countries, whether you like it or not. It has as much potential as winning the search engine race in early 2000s. The company who will manage to secure the market will turn into another Google in 15-20 years after that.

[-]

Confident_Ideal_5385@reddit

A quarter of OpenAI is owned by Microsoft. If Microsoft can burn money, so does OpenAI. Both Google and Amazon are investing into Anthropic, they have the money to burn too. If any of them run out of money, their investors can always give more.

This take shows me quite clearly that you didn't live through the 1998-2000 boom/bust cycle. Investors always have more money, right up until they don't.

[-]

No-Refrigerator-1672@reddit

This take shows me quite clearly that you didn't live through the 1998-2000 boom/bust cycle.

Neither did you. The reason why so much of dot com companies failed is because none of them had monetization paths. What do AI providers have? The clear path to monetization. See the difference?

[-]

temperature_5@reddit

Except 50% of data centers are on hold for lack of power, transformer hardware, etc. Eventually that will hit quarterly profits and estimates, causing a repricing of a huge swatch of AI and related companies.

Don't know about OP, but I did live through the dotcom bust, and it wasn't because the Internet wasn't a good idea, it was because there were a lot extreme valuations and only so many investors and consumers to prop them up. The real world always manages to slow adoption somehow.

[-]

Confident_Ideal_5385@reddit

The biggest issue with any new tech is not being able to see the difference between commercially sensible adaptations (buy books on the internet! Porn!) and fucking idiocy (pets.com. ten billion miles of unused single mode fibre.)

We're in the same stage here with transformer tech. Give it a bust cycle and 10 years and she'll be laughing.

[-]

Confident_Ideal_5385@reddit

The reason why so much of dot com companies failed is because none of them had monetization paths. What do AI providers have? The clear path to monetization. See the difference?

No, because you're just making that assertion without an iota of proof, whereas I've argued exactly why I see AI companies in 2026 as being exactly as profitable as 1999 .com startups. You're also acting like a whiny bitch so I'm not gonna engage further.

Have a nice day.

[-]

TheDeviceHBModified@reddit

You're arguing with someone who unironically uses "clanker". Don't feed the troll.

[-]

Nyghtbynger@reddit

Securing what ? Buying a few nvidia/amd cards, putting VLLM on it and GLM 5.1 then hiring a few data engineers ?

[-]

No-Refrigerator-1672@reddit

If it's that easy, then why don't you own the world's leading AI provider? Go for it, you'll be a billionaire!

[-]

TheDeviceHBModified@reddit

the clanker

Fuck off and skulk back to r/antiAI

[-]

ShelZuuz@reddit

You can absolutely host any model profitably on a Grace Blackwell and presumably Vera Rubin. NOT from $20/month subscriptions, but for the tokens the sell to enterprises, absolutely.

[-]

Nyghtbynger@reddit

They are going to have their asses kicked by logistics

[-]

05032-MendicantBias@reddit

The tokens they sell are incredibly valuable, have a nearly infinite TAM

The Total Addressable Market for current LLM and Diffusion inference is less than 100 billion dollars a year. And that's being generous.

You need specialized tools to be consolidated into professional software product like Photoshop to specialize them, but it just isn't going to happen that a generic LLM will absorb all the professional work across all domains. We are really, REALLY far from this.

[-]

Caffdy@reddit

there are thousands of data centers being built as we speak, you'll get tired of waiting because this balloon is just climbing up

[-]

fantasticsid@reddit

Yeah, looks real fuckin great for them.

[-]

temperature_5@reddit

Would be nice if everyone just standardized on this, including the consumer market.

[-]

Sea_Calendar_3912@reddit

this is great news. ram production news with latest technology leaps is always good news. it will eventually trickle down

[-]

05032-MendicantBias@reddit

This is great news.

Once the bubble pops, for a while we'll get flooded with SOCAMM2 kits and motherboards that can accept SOCAMM2 modules. I'm getting my wallet ready to buy that "e-waste" :D

[-]

dark-light92@reddit

There is no popping the bubble. A few companies will go bankrupt but the demand for compute will not decrease.

[-]

05032-MendicantBias@reddit

The reason I am certain there will be a bubble pop is because the silicon deployment exceed grid capacity.

Lots of silicon will have to be liquidated if only because there is no way to power it. And that's before considering online models are subsidized by venture capital, and that capital will run out.

The signal that herald the collapse is online providers going IPO while unprofitable. SpaceX+xAI+Twitter, OpenAI and Anthropic I suspect won't be around by this time next year, as they are going IPO in the second quarter of 2026.

And the hardware they bought, will be liquidated.

[-]

bick_nyers@reddit

Lack of power supply is a forcing function to purchase newer GPUs that are more power efficient.

When they start taking down the glut of Hopper in order to power Vera Rubin/Feynman etc., then maybe.

However if they are able to sell their H100s back to NVIDIA etc. to get refurbished/repackaged/limited then resold to China who has no such power constraints then I think the power constraints argument falls apart.

In 2025 China added 40% of the total US power grid in new power. They are not slowing down. They don't care if the GPUs are power inefficient.

Also, I know everyone thinks these are heavily subsidized by VC, and they are subsidized, but I don't think to the degree to which people think (people seem to think ~80% subsidy).

You can do the math on these things if you estimate the number of active parameters of the closed source models. The math isn't that far off from penciling as many would assume.

[-]

dark-light92@reddit

In principle I agree with you. But what stops remaining player from gobbling up the already built infrastructure & capacity? It's clear that the big corps want compute as a rented resource than a consumer owned one.

[-]

unspecified_person11@reddit

Demand for the internet didn't go away after the dotcom bubble, demand for houses didn't go away during the housing bubble.

It's never been about demand.

[-]