AMD crafts custom EPYC CPU for Microsoft Azure with HBM3 memory – CPU with 88 Zen 4 cores and 450GB of HBM3 may be repurposed MI300C, four chips hit 7 TB/s

Posted by ValidPrestige@reddit | hardware | View on Reddit | 21 comments

[-]

AbhishMuk@reddit

Got it, adding more bandwidth right away.

Not that it’ll fix latency, but hey, it’ll at least be fun!

[-]

COMPUTER1313@reddit

An IGP of a 7900XTX that shares the L4 HBM with a 9800X3D.

And since it's an APU, that means no pesky I/O die...

[-]

Brisngr368@reddit

Even if there was an I/O die you're still gonna have way lower latency than the PCIe bus

[-]

Arbiter02@reddit

Holy shit that’s a lot of HBM. We’ve certainly come a long way from early Volta and Vega topping out at 16gb

[-]

Does anyone know what the use case for these CPUs which HBM is? I see some Xeons with HBM on the Intel cloud and they are supposedly for inference (I guess batch size 1?) but I cannot figure out the cost structure where the lower latency makes sense, they seem expensive.

[-]

Verite_Rendition@reddit

There are certain classes of workloads that aren't embarrassingly parallel enough to benefit from GPUs, but do require a large amount of memory bandwidth all the same. These are often HPC, simulation, and engineering packages, such as Ansys.

It's basically the same rationale for why the HBM-equipped Xeon Max exists.

[-]

ScepticMatt@reddit

96 cores each:
https://www.servethehome.com/this-is-the-microsoft-azure-hbv5-and-amd-mi300c-nvidia/

[-]

randomkidlol@reddit

it sounds more like an AMD semicustom product rather than a repurposed MI300C. console APUs are not the only thing AMD semicustom produces.

[-]

Noble00_@reddit

Whether or not AMD or Microsoft will admit it being a repurposed MI300C (internally, they call it the AMD EPYC 9v64H), the MI300C has been for a long time rumoured, popping up in docs. Anyways, semicustom or not, like the PS5 Pro, codenamed "Violia", there isn't an upcoming product that won't be in the rumour mill. In fact, Dr. Ian Cutress has seemingly confirmed it being the MI300C, after all, it's on the same MI300 platform.

[-]

randomkidlol@reddit

might be a bit less semicustom than i thought if AMD is already planning to release a very similar product. that being said, this is microsoft we're talking about. paying for a custom chip on to be used on one of the largest cloud providers in the world doesnt seem odd.

[-]

uzzi38@reddit

AFAIK MI300C won't see a general release due to a lack of customer demand (and the relatively massive demand for MI300X - the two products require the same packaging and platform so they kind of compete with each other).

This is probably the only place we'll see MI300C, unfortunately.

[-]

Noble00_@reddit

Yeah, when this was going around and never really acknowledged by AMD, it was clear how niche this product was, and may have never seen the light of day (IIRC there was some talks of it being a China only product at the time). Tbf to the semicustom point, the platform features NVIDIA Quantum-2 InfiniBand and Azure Boost NICs, so not really an all AMD deployed platform.

[-]