Nvidia Takes an Added Role Amid AI Craze: Data-Center Designer

He is trying to build Nvidia into more than a supplier of a valuable hardware component: a one-stop shop for all the key elements in the data centers where tools like OpenAI’s ChatGPT are created and deployed, or what he calls “AI factories.”

Huang emphasized Nvidia’s growing prowess at data-center design following an earnings report Wednesday that exceeded Wall Street forecasts. The report came days after rival Advanced Micro Devices agreed to pay nearly $5 billion to buy data-center design and manufacturing company ZT Systems to try to gain ground on Nvidia. 

“We have the ability fairly uniquely to integrate to design an AI factory because we have all the parts,” Huang said in a call with analysts. “It’s not possible to come up with a new AI factory every year unless you have all the parts.”

It is a strategy designed to extend the business success that has made Nvidia one of the world’s most valuable companies—and to insulate it from rivals eager to eat into its AI-chip market share, estimated at more than 80%. Gobbling up more of the value in AI data centers both adds revenue and makes its offerings stickier for customers.

Nvidia is building on the effectiveness of its 17-year-old proprietary software, called CUDA, which enables programmers to use its chips. More recently, Huang has been pushing resources into a superfast networking protocol called InfiniBand, after acquiring the technology’s main equipment maker, Mellanox Technologies, five years ago for nearly $7 billion. Analysts estimate that InfiniBand is used in most AI-training deployments. 

You may also like

Up Next

0:00

ADVERTISEMENT

0:00

Paused

0:06 / 7:05

Tap For Sound

Nvidia is also building a business that supplies AI-optimized Ethernet, a form of networking widely used in traditional data centers. The Ethernet business is expected to generate billions of dollars in revenue within a year, Chief Financial Officer Colette Kress said Wednesday.

More broadly, Nvidia sells products including central processors and networking chips for a range of other data-center equipment that is fine-tuned to work seamlessly together. And it offers software and hardware setups catered to the needs of specific industries such as healthcare and robotics.

“He has verticalized the company,” Raul Martynek, CEO of data-center operator DataBank said of Huang’s strategy. “They have a vision around what AI should be, what software and hardware components are needed to make it so users can actually deploy it.”

Competition is growing

Nvidia’s competitors are responding. The Advanced Micro Devices deal to buy ZT Systems is fundamentally about gaining skills in how data centers are built. AMD plans to sell ZT’s manufacturing operations and retain its design experts to mount a more formidable challenge to Nvidia.

AMD has made other acquisitions to bolster its data-center offerings in recent years, including of programmable-chip company Xilinx and data-center networking company Pensando in 2022.

Other chip suppliers, from Intel to AI-chip startups such as Cerebras Systems and SambaNova Systems, are offering services and systems that do a large share of the work for customers who want to build and operate AI tools.

Analysts and industry executives say the trend partly reflects a customer preference for plug-and-play AI computing infrastructure that helps them move quickly amid an AI boom where speed is paramount.

Doing more of the heavy lifting may help chip-makers better compete as AI investment shifts from experienced tech giants such as Meta Platforms and Microsoft to a wider swath of businesses with less expertise.

“Enterprises typically go with the turnkey solution,” said Srini Pajjuri, an analyst at investment firm Raymond James. “They don’t have the resources, they don’t have the technology or know-how” to cobble together the equipment themselves, he said.

Risks in the strategy

There are some risks in the strategy, Pajjuri said. As AI computing becomes more competitive and the rush to set up AI factories subsides, customers might explore alternatives to many of the proprietary technologies that Nvidia has built up around its AI chips.

“Right now, time to market is the most important thing, so they’re adopting solutions from Nvidia, but as the technology matures, Nvidia might not be able to capture more of the pie,” he said.

Nvidia also could confront regulatory hurdles that have sprung up for many companies in the past that have come to dominate a market and extend their influence in it. The company is under scrutiny for its market practices in Europe, and its offices in France were raided this past year, although no charges have been brought against the company there.

On Wednesday, Huang described Nvidia’s role as one of a coordinator and designer that could give guidance on how to set up complex AI infrastructure, but he emphasized that Nvidia isn’t going to do everything.

With its next-generation Blackwell AI chips, set to ship out late this year or early next year, it is offering a design for an entire rack of computing equipment for the first time—a cabinet about 2 feet wide and 6 feet high. But Nvidia itself won’t make them.

“We know how to design the AI infrastructure, provide it the way that customers would like it and let the ecosystem integrate it,” Huang said.

Write to Asa Fitch at asa.fitch@wsj.com

Making of a Tech Giant

How Nvidia's growth has rocked markets

The Investors Who Found Nvidia Early

What Nvidia, Apple Have in Common

Nvidia’s Success Is the Market’s Problem

The Man in His 80s Who Saved Nvidia