Nvidia's 900 tons of GPU muscle bulks up server market, slims down wallets

The server marketplace for nan adjacent early is going to beryllium astir GPUs, GPUs, and much GPUs, according to Omdia. The marketplace interrogator estimates nan measurement of Nvidia H100 GPUs unsocial shipped during almanac Q2 added up to much than 900 tons successful weight.

This headline-grabbing fig comes from nan company's latest Cloud and Data Center Market Update, which notes nan displacement successful datacenter finance priorities it previously highlighted continues apace. The study says request for servers fitted pinch 8 GPUs for AI processing activity has had nan effect of pushing up mean prices, while simultaneously putting nan compression connected finance successful different areas.

As a consequence, Omdia has erstwhile again lowered its estimate for yearly server shipments for 2023 by different 1 cardinal units to 11.6 million, a 17 percent diminution complete past year's figure. At nan aforesaid time, however, nan mean value of a server has risen by much than 30 percent quarter-on-quarter and year-on-year because hyperscalers are pouring finance into these high-spec boxes.

The study states that conscionable complete 300,000 of Nvidia's H100 GPUs recovered their measurement into nan assembly lines of server makers, pinch nan 900 tons fig based connected each GPU pinch its heatsink weighing successful astatine astir 3kg.

With each H100 carrying an eye-watering value tag of astir $21,000 each, this paradoxically intends that Omdia now expects full server marketplace gross for 2023 to travel successful astatine $114 billion, up 8 percent year-on-year, contempt portion shipments being down markedly.

This immense request for AI servers bristling pinch GPUs is mostly driven by nan hyperscale companies and unreality providers, according to Omdia, pinch nan beneficiaries being nan original creation shaper (ODM) server makers – nan alleged achromatic container vendors.

It claims that a ample fraction of those eight-GPU servers shipped successful nan 2nd 4th was built by ZT Systems, a US-based unreality server shaper that is described arsenic a fast-growing, privately owned company. Evidence for this is based connected a declare that arsenic overmuch arsenic 17 percent of Nvidia's $13.5 cardinal second 4th revenue was driven by a azygous vendor, and that Nvidia CEO Jensen Huang showcased a DGX server manufactured by ZT Systems during Computex Taipei.

Omdia besides believes that overmuch of nan maturation successful request for eight-GPU servers during Q2 was apt down to Meta, based connected Nvidia antecedently saying that 22 percent of nan quarter's gross was driven by a azygous unreality work provider.

• Mention AI successful net calls ... and watch that stock value leap
• Morgan Stanley values Tesla's super-hyped supercomputer astatine up to $500B
• d-Matrix bets connected in-memory compute to undercut Nvidia successful smaller AI models
• Now Middle East nations banned from getting top-end Nvidia AI chips

And location is much of this to come. The study predicts nan accelerated take of servers configured for AI processing will proceed for nan 2nd half of this twelvemonth and nan first half of 2024.

However, Omdia forecasts a better-than-seasonal rebound successful request for general-purpose servers during nan existent 3rd 4th and nan adjacent arsenic endeavor request picks up. This will spot gross successful 3Q23 up by 13 percent year-on-year, accelerating to 29 percent successful 4Q23, it predicts.

In particular, Omdia expects nan continued deployment of eight-GPU servers to consequence successful server marketplace gross increasing by 51 percent year-over-year during nan first half of 2024, pinch a cardinal H100 GPUs forecast to find their measurement into systems.

Yet nan expert cautions that nan accelerated finance successful AI training capabilities it is seeing among hyperscalers and unreality providers should not beryllium confused for accelerated adoption. AI usage is still mostly low, it said, citing a study that recovered that only 18 percent of US adults person utilized ChatGPT. Generative AI besides presently makes up only a mini fraction of unreality computing costs for enterprises arsenic well, arsenic The Register reported recently.

There are besides downsides to this immense finance successful power-hungry GPU systems for AI training. One media manufacture exec precocious warned that AI will "burn nan world" unless extremist action is taken to instrumentality much sustainable practices.

Speaking astatine nan International Broadcasting Convention (IBC) successful Amsterdam this month, Ad Signal CEO and laminitis Tom Dunning said that while we can't inquire organizations to springiness up nan commercialized benefits of AI, it is clear that "there's an urgent request to minimise its c impact."

Datacenters are now responsible for 2.5 to 3.7 percent of each c dioxide emissions, Dunning claimed, and take of AI is projected to turn astatine a 37 per cent compound yearly maturation complaint (CAGR) betwixt now and 2030. ®