Hyperscalers getting ready for AI workloads are buying up vast amounts of cloud infrastructure hardware.
Spending on compute and storage infrastructure for cloud deployments grew 18.5% year over year in the fourth quarter of 2023, according to the latest update to the IDC Worldwide Quarterly Enterprise Infrastructure Tracker, which was issued last week. The total number of units shipped declined, however, which reflects hyperscalers’ appetite for high-capacity, GPU-heavy servers with higher average selling prices.
AI is behind the steep rise in cloud infrastructure spending, according to Lidice Fernandez, a group vice president at IDC and one of the co-authors of the report. More particularly, it’s the rapid build-out of AI capability by some of the biggest companies in the technology sector, including major hyperscalers like Google and Microsoft, coupled with the fact that the hardware needed to cope with AI-centric workloads involves expensive GPU hardware.
“Those large, large, mega-datacenters are the ones that swing volumes,” she said. “So this year, we started seeing all of these large hyperscalers and cloud service providers favor GPU-rich configurations, which tend to be more expensive in processing capacity.”
Simultaneously, however, those expensive, GPU-heavy devices are capable of handling heavier workloads, meaning that these companies need fewer of them to create a given amount of capacity, Fernandez added.
“You probably need fewer boxes to accomplish the same workload, so that the dynamics we’re seeing, in terms of the largest buyers of infrastructure, will continue exponentially for only a couple of years,” she said.
In terms of raw data, the fourth quarter of 2023 saw sales numbers for cloud infrastructure products rise 18.5% year-over-year, to a total of $31.8 billion. That’s higher than spending on non-cloud compute and storage infrastructure, according to IDC’s data, which rose by 16.4% year-over-year, to $18.9 billion. The total number of individual shipments of cloud equipment dropped by 22.8% during the same time period.
The high demand among hyperscalers and major service providers also skewed the data towards the public cloud side of the equation, IDC’s data showed. “Shared” cloud infrastructure, which largely covers public cloud-type deployments, saw 27% growth compared to a year earlier, reaching $22.8 billion. That also represents nearly 45% of total worldwide infrastructure spending, cloud or non-cloud. By contrast, “dedicated” cloud, which is mostly private cloud infrastructure, grew by a mere 1.4% over the same time frame, to $9 billion.
“Most of the spending continues to be concentrated on the public cloud area,” Fernandez said.
Don’t count out private cloud growth
Nevertheless, Fernandez noted, private cloud will remain important for the foreseeable future, not least because the rank and file of cloud users – not the hyperscalers and service providers – has a significant and potentially growing need for it.
“Everything that remains [on-premises or in private cloud] is there for a reason,” she said. “In many cases, the reasons are privacy, compliance and security.”
Plenty of workloads that are mission-critical – particularly those subject to industry specific regulation or new general regulations in jurisdictions like the EU – have not only been kept in private or on-prem deployments, but many that have been moved to the public cloud are being “repatriated,” Fernandez said.
“If you see the numbers overall, they remain relatively stable in terms of spending,” she said. “That infrastructure needs to continue being sustained and deployed and refreshed at a certain pace.”
