Announcing Azure HBv5 Virtual Machines: A Breakthrough in Memory Bandwidth for HPC | Microsoft Community Hub

Posted by Noble00_@reddit | hardware | View on Reddit | 4 comments

[-]

Noble00_@reddit (OP)

The rumored MI300C making an appearance?

Each Azure HBv5 VM will feature:

6.9 TB/s of memory bandwidth (STREAM Triad) across 400-450 GB of RAM (HBM3)
Up to 9 GB of memory per core (customer configurable)
Up to 352 AMD EPYC “Zen4” CPU cores, 4 GHz peak frequencies (customer configurable)
2X total Infinity Fabric bandwidth among CPUs as any AMD EPYC™ server platform to date
SMT disabled, single-tenant only design (1 VM per server)
800 Gb/s of NVIDIA Quantum-2 InfiniBand, balanced as 200 Gb/s per CPU SoC
Azure VMSS Flex to scale MPI workloads to hundreds of thousands of HBM-powered CPU cores
160 Gbps of Azure Accelerated Networking via 2nd generation Azure Boost NIC
14 TB local NVMe SSD delivering up to 50 GB/s read and 30 GB/s write bandwidth

[-]

blaktronium@reddit

You could run decently performant virtual machines using only that disk for vmem and not using any host memory at all. Wild.

[-]

That's sequential performance on the SSDs, i assume. So unless you REALLY optimize your "memory access" to hit cache heavily, you won't even get close to that 30-50 GB/s bandwidth. Probably 10-100x slower.

[-]

coniurare@reddit

Looking forward to people hopefully testing the memory latency. At a quick glance, the HBM from Saphire Rapids appears to have a latency of ~134ns compared to ~106ns for its DDR5 variant. GDDR6 on AMDs 4700s (PS5 rejects) seems to have a latency of ~145ns.