AMD releases details of 288GB MI355X accelerator: 80% faster than MI325X, 8TB/s memory bandwidth

Rival to Nvidia Blackwell has some tasty specs, but will it be enough to catch up with Nvidia?

· TechRadar

News By Desire Athow published 10 October 2024

(Image credit: AMD)

We already knew a lot about AMD’s next generation accelerator, the Instinct MI325X, from an earlier event in June 2024 - but the company has now revealed more at its AMD Advancing AI event.

First, we knew the Instinct MI325X was a minor upgrade from the MI300X, with the same CDNA 3 architecture, but just enough oomph to make it a viable alternative to the H200, Nvidia’s AI powerhouse.

Eagle-eyed readers will also notice that AMD has cut the onboard HBM3e memory capacity from 288GB to 256GB with the memory capacity now only 80% more than Nvidia’s flagship rather than the more enviable 2x improvement.

Preparing the grounds for the MI355X

To make things a bit more murkier, AMD also mentioned another SKU, the MI325X OAM which will have, wait for it, 288GB memory - we have asked for clarification and will update this article in due course.

AMD provided some carefully selected performance comparisons against Nvidia’s H200:

  • 1.3X the inference performance on Mistral 7B at FP16
  • 1.2X the inference performance on Llama 3.1 70B at FP8
  • 1.4X the inference performance on Mixtral 8x7B at FP16

The company also revealed the accelerator has 153 billion transistors, which is the same as the MI300X. The H200 has only 80 billion transistors while Blackwell GPUs will top the scale at more than 200 billion transistors.

The star of the show though had to be the MI355X accelerator, which was also announced at the event with an H2 2025 launch date. Manufactured on TSMC’s 3nm node and featuring AMD’s new CDNA 4 architecture, it introduces FP6 and FP4 formats and is expected to deliver improvements on 80% on FP16 and FP8, compared to the current MI325X accelerator.

Are you a pro? Subscribe to our newsletter

Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!

Contact me with news and offers from other Future brandsReceive email from us on behalf of our trusted partners or sponsors