Bain warns: Prepare for AI chip shortage

Surging demand for AI computing power will strain the supply chains for data center chips, personal computers, and smart phones, and, combined with “continued geopolitical tensions and other supply risks, could trigger the next semiconductor shortage,” a report released Tuesday by Bain & Company stated.

Authors of the report suggested that while semiconductor suppliers could not have predicted the pandemic, they could guard against the next big threat to semiconductor supply chains.

They noted that the “supply and demand of semiconductors is a delicate balance, as the industry and its customers know all too well after the past few years. Although the pandemic-induced chip shortage has passed, executives are starting to prepare for the next potential crunch caused by (you guessed it) artificial intelligence.”

The semiconductor supply chain, the report said, is “incredibly complex, and a demand increase of about 20% or more has a high likelihood of upsetting the equilibrium and causing a chip shortage.”

Key findings revealed that:

• Spending on data centers (DCs) and the specialized chips that power them shows no signs of slowing. Major cloud service providers are expected to increase their year-over-year capital spending by 36% in 2024, spurred in large part by investments in AI and accelerated computing.
• If data center demand for current-generation graphics processing units (GPUs) doubled by 2026 — a reasonable assumption given the current trajectory — suppliers of key components would need to increase their output by 30% or more in some cases.
• To enable artificial intelligence (AI) growth, “a complex web of supply chain elements must come together, from constructing data centers and wafer fabs (fabrication facilities) to securing access to advanced packaging and sufficient electricity.”

While the focus of the report is on what organizations who buy chips need to do, there are steps CIOs can take to ensure they have access to the products they need, or prepare for dramatic shifts in price points.

According to Scott Bickley, research practice lead at Info-Tech Research Group, “the advanced semiconductor supply chain is the most fragile supply chain on the planet. Literally over 5,000 vendors must work in perfect harmony to produce the most advanced chips.”

Many of these vendors, he said, “supply a single component to a single company, without which the whole system comes screeching to a halt. The technical obstacles alone are mind-boggling, notwithstanding the geopolitical risks facing TSMC and the normal headwinds of logistics management.”

Technology buyers, said Bickley, are “divided into two key buyer segments: those procuring for an infrastructure layer at scale, such as a private cloud environment … or, say, an F200-sized consumption customer, and those purchasing for their smaller-scale projects, such as data center modernization, small-scale LLM in-house models, and advanced AI-enabled PCs.”

At the private cloud layer, “buyers must formulate a technology strategy now. Are you going for the gusto, for example, and placing bets on Nvidia’s next-gen Blackwell line of GPUs, or are you going to purchase the first-generation H100 and spend more time training your model as an alternative? The DC infrastructure obstacles are non-trivial, with the shift to water-cooled environments and the necessity of engineering high-density GPU clusters that balance energy consumption, performance, and environmental mandates.”

According to Bickley, the “traditional tech buyer in an enterprise environment has a different set of challenges, primarily due to their lack of scale, limiting their supplier influence. Buyers in this environment will have to over-extend and make some bets now to secure supply later. Providers of the newest AI-enabled PCs and servers, which will flow through traditional providers such as Dell and Supermicro, for example, are interested in serious order placers now.”

Planning ahead for delays in production may require buyers to take on some expensive inventory of bleeding-edge technology products that may become obsolete in short order, he said.

Alvin Nguyen, senior analyst at Forrester Research, said when it come to what CIOs can do to ensure they continue to have access to the products they need, or prepare for dramatic shifts in pricing, there are several categories they will need to consider:

 These include:

• Risk: The rate of progress in generative AI (genAI) and heavy investment in a specific model or approach today may prove to be the wrong or non-optimal choice later, said Nguyen:  “For most enterprises that seek to leverage AI as opposed to driving the AI market, being risk adverse and leveraging existing AI services instead of making a big push towards acquiring significant AI infrastructure makes the most sense.”
• Personnel:  CIOs and tech execs, he said, “need to invest in training/upskilling of existing staff and hiring of new AI-ready resources for basic AI skills with known AI use cases that can be leveraged effectively, such as code development. They need their technologists, architects, and engineers to experiment with the latest AI technology to determine the choices they need to make.  If you are able to acquire significant AI infrastructure, then invest heavily here to develop a competitive advantage over others.”
• AI infrastructure: servers and AI accelerators: Demand, said Nguyen, exceeds supply for the time being, and this “is unlikely to change over the next few years, so there will be a premium for AI accelerators/GPUs for the time being.  Unless you are able to get large enough allocations of AI accelerators, leveraging AI services from AI and cloud service providers will make more sense.”
• Sustainability: Nguyen said that genAI’s need of more energy and water resources, along with its carbon footprint, has impacted the ability of some organizations to meet their sustainability goals. 
  “[This is unlikely] to change while demand for AI continues to grow,” he said. “CIOs and tech execs need to purchase power from renewable sources and employ sustainable building and operating practices (choice of building materials, construction methods, recycling) where possible to get back on track.”