The upcoming MI300, which will ship Latter this year after NVIDIA's Grace/Hopper Superchip, certainly has a shot at it. But there remain a lot of unknowns that will determine how well it performs for AI applications. And then there is Software. AMD Instinct MI300 is a combination of its flagship CPU and GPU. At the 2023 CES Keynote address, AMD's CEO Dr. Lisa Su reiterated the company's plan to bring the Instinct MI300 to market by the end of this year, and showed the monster silicon in hand. The chip is certainly a major milestone for the company, and the industry in general, being to most aggressive chiplet implementation seen so far.

Everything we know about memory requirements in machine learning may be wrong. Today, when data scientists process deep learning models using a "throughput computing" device like a GPU, TPU, or similar hardware accelerator, they're likely faced with a decision to shrink their model or input size to fit within the device's memory limitations. Training a large, deep neural network (or even a wide, shallow one) on a single GPU, in many cases, may be impossible. Ever wonder why on the original Resnet 152, the winner of the ILSVRC-2015 image detection competition had 152 layers and not 153? Is it a coincidence that the parameters in 152 layers have a memory footprint of slightly less than 12G, while 153 layers go beyond 12G (the standard size of GPU memory at the time)?