Today, NVIDIA posted the fastest results on new MLPerf benchmarks measuring the performance of AI inference workloads in data centers and at the edge. The new results come on the heels of the company's equally strong results in the MLPerf benchmarks posted earlier this year. MLPerf's five inference benchmarks -- applied across a range of form factors and four inferencing scenarios -- cover such established AI applications as image classification, object detection and translation. NVIDIA topped all five benchmarks for both data center-focused scenarios (server and offline), with Turing GPUs providing the highest performance per processor among commercially available entries. Xavier provided the highest performance among commercially available edge and mobile SoCs under both edge-focused scenarios (single-stream and multistream).
Nvidia is touting another win on the latest set of MLPerf benchmarks released Wednesday. The GPU maker said it posted the fastest results on new MLPerf inference benchmarks, which measured the performance of AI inference workloads in data centers and at the edge. MLPerf's five inference benchmarks, applied across four inferencing scenarios, covered AI applications such as image classification, object detection and translation. Nvidia topped all five benchmarks for both data center-focused scenarios (server and offline), with its Turing GPUs. Meanwhile, the Xavier SoC turned in the highest performance among commercially available edge and mobile SoCs that submitted for MLPerf under edge-focused scenarios of single-stream and multi-stream.
After introducing the first inference benchmarks in June of 2019, today the MLPerf consortium released 595 inference benchmark results from 14 organizations. The MLPerf Inference v0.5 machine learning inference benchmark has been designed to measure how well and how quickly various accelerators and systems execute trained neural networks. The initial version of the benchmark, v0.5 currently only covers 5 networks/benchmark, and it doesn't yet have any power testing metrics, which would be necessary to measure overall energy efficiency. In any case, the benchmark has attracted the attention from the major hardware vendors, all of whom are keen to show off what their hardware can do on a standardized test, and to demonstrate to clients why their solution is superior. Of the 595 benchmark results released today, 166 are in the Closed Division intended for direct comparison of systems.
"If all 7.7 billion people on Earth uploaded a single photo, you could classify [them all] in under 2.5 hours for less than $600" Google and Nvidia have both declared victory for their hardware in a fresh round of "MLPerf" AI inference benchmarking tests: Google for its custom Tensor Processing Unit (TPU) silicon, and Nvidia for its Turing GPUs. As always with the MLPerf results it's challenging to declare an overall AI leader without comparing apples with oranges: Alibaba Cloud also performed blisteringly strongly in offline image classification. The MLPerf Inference v0.5 tests were released Wednesday, and capture some of the impressive performance taking place in AI/machine learning, both at the hardware and software level. These performance improvements are rapidly filtering down to the enterprise level: as sophisticated customer service chatbots, models to predict investment outcomes, to underpin nuclear safety, or discover new cures for disease. The MLPerf tests are designed to make it easier for companies to determine the right systems for them, as machine learning silicon, software frameworks and libraries proliferate.
The first benchmark results from the MLPerf consortium have been released and Nvidia is a clear winner for inference performance. For those unaware, inference takes a deep learning model and processes incoming data however it's been trained to. MLPerf is a consortium which aims to provide "fair and useful" standardised benchmarks for inference performance. MLPerf can be thought of as doing for inference what SPEC does for benchmarking CPUs and general system performance. The consortium has released its first benchmarking results, a painstaking effort involving over 30 companies and over 200 engineers and practitioners.