Nvidia launches Spectrum-X Ethernet stack for AI workloads

Nvidia on Tuesday launched Spectrum-X, its latest generation of Ethernet networking technology, designed for data centers with heavy AI workloads.

Spectrum-X, which integrates into Nvidia’s Spectrum Ethernet stack, is a combination of hardware and software, melding the Spectrum-4 Ethernet switch (which boasts up to 51 terabits per second) and BlueField-3 DPU together, which are designed to work in tandem to reduce traffic congestion and potentially eliminate packet loss.

This new stack, according to Nvidia, offers a 1.6x improvement over traditional Ethernet for AI applications, a necessity as AI workloads generate significantly more data and can quickly congest data centers.

Nvidia calls its AI-enabled network interfaces a SuperNIC, a specialized network accelerator designed for AI cloud computing, providing high-speed Ethernet connectivity and efficiency enhancements for hyperscale AI workloads.

The new technology will be available in servers from Dell, Hewlett Packard Enterprise (HPE), and Lenovo. The new servers also include Nvidia’s latest H100 Tensor Core GPUs and its AI Enterprise, as well as AI Workbench software.

“Through our collaboration, Dell Technologies and Nvidia are providing customers with the infrastructure and software needed to quickly and securely extract intelligence from their data,” Dell Chairman and CEO Michael Dell said in a release.

Nvidia is best known for its GPUs, but the company has also made network switches, with the Spectrum series first being known for accelerating big data workloads in data centers.

“Generative AI will undoubtedly drive innovation across multiple industries,” said Antonio Neri, president and CEO of HPE, in a press release. “These powerful new applications will require a fundamentally different architecture to support a variety of dynamic workloads.”

Spectrum-X is currently used in Nvidia’s Israel-1 supercomputer, built with Dell servers and Nvidia’s HGX 100 platform, featuring GPUs, BlueField-3 DPUs, SuperNICs, and Spectrum-4 switches. According to Nvidia, this model serves as a reference for firms building AI-driven high-performance computing clusters.

Broadcom says “not new”

Nvidia first announced Spectrum-X at the Computex trade show in Taipei, with CEO Jensen Huang saying on stage that it was “the world’s first high-performance ethernet for AI” and it doesn’t drop packets.

But at the time, Broadcom shot back, telling The Register that “there’s nothing unique about their device that we don’t already have.”

Ram Velaga, SVP of Broadcom’s core switching group, told The Register that Broadcom’s technology already effectively manages congestion in a more vendor-agnostic way, taking aim at Nvidia’s proprietary software layer.

“The whole reason why Ethernet is successful today is it’s a very open ecosystem,” he is quoted as saying.

AI throughout the data center

Demand for new AI-ready network infrastructure, such as Nvidia’s Spectrum-X, is largely being driven by a surge in data center spending to build capacity to handle these workloads. 

In Q2 2023, global spending on cloud infrastructure products increased by 7.9% to $24.6 billion, research firm IDC said in a release. 

IDC said that it expects worldwide spending on AI solutions will grow to more than $500 billion in 2027, with 41% of retail banking in Asia inclined to invest in it, and 80% of CIOs will leverage organizational changes to harness AI, both driving demand for AI-ready data centers.

Looking towards earnings

Despite the cloud of export curbs to China threatening to dampen Nvidia’s earnings, the company’s stock has topped $500 – an all-time high – largely because of its position of being well-capitalized on continued demand from the AI revolution pushing spending on data centers.

“We believe demand from US cloud and other data center clients remains strong and intact given these firms are still in the process of transforming their data centers with accelerated compute capabilities,” Piper Sandler analyst Harsh Kumar wrote in a recent note.