AI’s real bottleneck isn’t compute — it’s the network underneath

For such a rapid innovation cycle, AI has been given unprecedented levels of responsibility. According to the Stanford AI Index 2026, 88% of organizations used AI in 2025, with 70% using generative AI in at least one business function. Most analysts agree that this technology, particularly when it comes to generative and agentic AI, is yet to reach full maturity, yet it’s already making itself indispensable to most enterprises. Employees expect LLM-based copilots to respond as readily as any other business application, customers are increasingly exposed to AI as part of their user experience, and emerging agentic models need to communicate continuously with applications and infrastructure as they crunch data and carry out tasks. Any hint of delay within those interactions has the potential to disrupt productivity, sow mistrust in the technology, and limit any return on investment (ROI).

Most businesses now inhabit a multi-cloud environment that spans countries and continents. When an AI request depends on information stored in one cloud environment, processing capacity hosted in another, and an application delivered somewhere else entirely, latency becomes the deciding factor. Each network hop, particularly through public Internet pathways, increases response time. Repeat these delays across hundreds, thousands, or even millions of individual requests – from both humans and AI agents – and the whole organization becomes artificially hampered. Sometimes connectivity becomes so hampered that IT teams must step in and spend valuable time rectifying it.

A recent Censuswide survey commissioned by DE-CIX found that IT teams in the US and UK spend an average of 11.5 hours every week resolving cloud connectivity issues – that’s more than a full working day each week spent on network troubleshooting. This is the “hidden productivity tax” that organizations are now paying, and it may be why the ROI for AI initiatives feels lukewarm at best, and impossible to measure at worst. McKinsey’s 2025 global AI survey found that only 39% of respondents could attribute any enterprise-level EBIT impact to AI, and most of those reported a contribution below 5%. That’s why even the best models can only get you so far – compute power may command the budget, but it’s the network that determines value.

A tax on productivity

Consider what happens when an employee asks an AI assistant to carry out a simple task like summarizing an internal document or doing some number crunching. The application may need to verify the employee’s identity, locate the relevant information, send it to a model operating in another cloud environment, and return the response through the user-facing application. An agent completing a more involved task could make this journey repeatedly as it gathers information and interacts with other systems. If one connection is slow or unreliable, the delay follows the request all the way back to the user, and if that connection fails, a whole team may be drafted in to isolate a problem that appeared inside the application but originated somewhere along the network path.

Engineers who could be improving AI services or supporting new deployments are instead tracing routes, investigating intermittent failures, and working around infrastructure that behaves unpredictably. What makes this particularly striking is the contradiction that was revealed as part of DE-CIX’s survey. While enterprises concede that more than a full working day per week is spent on network troubleshooting, as noted above, nearly all (96%) say their networks are “ready” to support future cloud and AI projects. This doesn’t prove that enterprise networks are fundamentally unprepared, but it does reveal a disconnect between strategic confidence and everyday operational experience. When more than a working day can disappear into connectivity troubleshooting each week, some of the expected productivity benefit from AI is already being spent before the technology has had an opportunity to deliver it.

AI performance and network performance are inseparable

Users don’t experience a model and its supporting infrastructure as individual sets of requests – they just experience “fast” or “slow” workloads. A cutting-edge AI model will still feel ineffective if the information it needs travels along congested or unpredictable public transit routes, so any real-time interaction with AI will become an exercise in frustration with small delays that compound over time, slowing teams down.  As AI services become more distributed, network performance is becoming inseparable from application performance, but it’s rarely given the same attention as model accuracy, compute capacity, or the cost of running AI inference.

IT leaders are beginning to take note, however, despite being a little overconfident about their network readiness. Alongside latency between cloud environments, security vulnerabilities including DDoS attacks, and downtime or reliability concerns are triggering a change in approach towards network connectivity. That’s why many are now turning to private, direct connectivity to cloud providers through an Internet or Cloud Exchange or via other means of directly connecting to the cloud providers’ private connectivity solutions, bypassing the pitfalls of the public Internet and IP transit.

Direct connectivity offers a greater degree of sovereignty, visibility, and control over data pathways that can reduce latency and enhance security. Public Internet connectivity will remain essential, of course, but business-critical AI traffic should not follow routes that can change unexpectedly or introduce additional network hops. It isn’t a universal cure for poor AI performance, but it can remove some of the uncertainty from an operating model that increasingly depends on information moving quickly and reliably between distributed environments.

Turning network performance into a business metric

The next 12 months could be a turning point for businesses looking to prove AI’s worth. CIOs simply need to account for the full cost of delivering an AI service, including the hours spent diagnosing connectivity problems and the value lost when applications respond inconsistently. In other words, they need to bring network teams into AI planning from the outset, considering their network infrastructure with the same rigor they apply to selecting AI models and projects. Private interconnection can provide more direct and predictable routes between environments, while effective observability can reveal how traffic moves, where delays accumulate, and how reliably the entire workflow performs. Without that control and visibility, network friction will remain a “hidden tax on productivity” that could stall AI deployments and weaken the case for future investment.