MLCommons, the artificial intelligence benchmarking group, recently unveiled a new set of tests and results that assess the performance of high-end hardware when running AI applications and responding to user queries. These tests comprise two fresh benchmarks that gauge the speed of AI chips and systems in generating responses from data-packed AI models. The results provide an approximation of how quickly an AI application, such as ChatGPT, can deliver a response to a user’s query.
One of the two new benchmarks, called Llama 2, measures the swiftness of a question-and-answer scenario for large language models and incorporates 70 billion parameters. Meta Platforms developed this benchmark. The second benchmark, which is part of the MLPerf suite of benchmarking tools, is a text-to-image generator based on Stability AI’s Stable Diffusion XL model.
Servers powered by Nvidia’s cutting-edge chips, created by companies such as Google, Supermicro, and Nvidia itself, easily won both the new benchmarks on raw performance. Several server builders submitted designs based on Nvidia’s less powerful L40S chip. Server builder Krai submitted a design for the image generation benchmark with a Qualcomm AI chip that consumes significantly less power than Nvidia’s leading-edge processors.
Intel also submitted a design based on its Gaudi2 accelerator chips, and the company described the results as “solid.” However, raw performance is not the only crucial factor when deploying AI applications. Advanced AI chips consume massive amounts of energy, and one of the primary challenges for AI firms is to deploy chips that provide optimal performance while consuming minimal energy. To address this, MLCommons has a separate benchmark category for measuring power consumption.
