The AI readiness gap: Why networks matter more than ever

Ask enterprise leaders about AI and you’re likely to get a wave of excited responses. BCG research found that two-thirds of global CEOs put accelerating AI among their top three priorities, with CIOs under pressure to turn that ambition into business value.

But there’s a problem. Many enterprise AI initiatives are struggling to move beyond pilots into production. Despite near-universal adoption, McKinsey finds that 88% of organizations now use AI in at least one business function, while almost two-thirds remain stuck in pilots and experimentation.  

“When it comes to AI readiness, most organizations are still trying to figure it out,” says industry expert Bill Burns. “We’re all asking the same questions: where should workloads live, how will traffic move, what does security look like, and where are the bottlenecks going to appear?”

The reasons are well documented, and most have nothing to do with infrastructure: unclear ROI, poor data quality, governance gaps, change-management fatigue, and a shortage of talent. Any honest account of why pilots stall has to start there.

But there is a common thread why these problems keep surfacing at the same companies, and it sits underneath all of them. Businesses can fix their data strategy, governance model, and talent pipeline, and still find that workloads won’t move where they need to, when they need to, at the cost they need. That constraint is the network – the one layer that gates whether the rest can actually run in production.

Why AI traffic is different and legacy networks can’t cope

Enterprise networks have always evolved to reflect changes in technology and working patterns. The rise of cloud computing and mobile devices in the mid-2000s, for example, shifted enterprise applications from the data center to public clouds and made the internet the network of choice.

AI is triggering the next major shift. It changes the shape, speed and economics of data movement, creating new traffic patterns that legacy infrastructure was never designed to handle. Unless networks adapt, AI will struggle to move beyond pilots into production.

The first challenge comes from training AI models. Unlike traditional enterprise traffic, AI workloads are persistent and continuous, creating demands that can overwhelm existing networks.

“The problem is that many of us are trying to modernize while still keeping the lights on,” says Burns. “It’s a pendulum every day between operational stability and preparing for what comes next.”

Training AI models requires data centers with high bandwidth, ultra-low latency and near-zero packet loss. Networks previously handling 100Gb may now need 400Gb or even 800Gb capacity. In distributed GPU clusters, one delayed packet can stall synchronization across thousands of dollars of compute resources in real-time.

The inference challenge

The second challenge comes from inference, where users interact with AI systems and AI agents talk to each other. This shifts traffic from north-south flows to far greater volumes of east-west machine-to-machine traffic, potentially increasing network demands by as much as 100x.

Furthermore, AI agents operate far faster than humans, meaning millisecond-level delays can become critical bottlenecks. As devices are increasingly used by both people and agents, enterprise networks will need to operate at machine speed.

“The network is no longer a foster child in the AI era,” says Murali Krishnan, associate vice president and head of the strategic products group for the Americas at Tata Communications. “It is the fabric – the epicenter around which performance, ROI and experience will be measured. CIOs need to unlearn what they knew about networks of the past, because how you design and deploy the network has changed from the ground up.”

What AI-ready networks look like

After the physical networks of the 1990s and the software-defined networks of the 2010s, we’re moving into the era of cognitive and contextual networks, fit for the unique requirements of AI. Static, best-effort infrastructure is giving way to networks that can observe, prioritize and adapt in real-time. We believe this new infrastructure must be built on three principles.

1. Unlike today’s enterprise networks, AI-ready networks will be natively intelligent and autonomous, with deep observability built in as standard. In AI environments, one delayed flow can ripple across an entire workload.

“Most networks can move AI traffic. The difference is whether they understand it,” says Rajat Gopal, vice president, cloud networking and security solutions at Tata Communications. “That means application awareness – knowing which workload a flow serves – consistency you can measure in jitter, not just an uptime number, and sovereignty enforced in the path itself, so data is geofenced by default.”

• Given enterprises’ hunger for data, IT leaders will need to architect their future networks with elasticity and scalability in mind – not just increased link capacity, but also more effective congestion domain boundaries and more controlled interconnect paths between clouds.

• Because the old perimeter-based security model is defunct in an era of AI-powered threats, when data moves continuously across domains, security and control have to be embedded into routing logic, not bolted on.

Those guiding principles start to map out a way for enterprises to prepare for AI at a foundational level. The network is becoming an active control plane for AI performance, cost and compliance. It also helps address some of the biggest headaches facing IT leaders currently, such as data sovereignty compliance (through visibility into data paths and metadata) and cost optimization (via lowering egress fees).

“We didn’t set out with AI in mind,” says Thor Wallace, CIO at NETSCOUT. “But as it turns out, the decisions we made through our digital transformation have put us in a position where we’re ready for it. The biggest driver was ensuring we had pervasive visibility across the network.”

The time to act

As AI agents spread, the network is becoming a critical – yet frequently overlooked – enabler of enterprise AI success.

The opportunity is significant. As Seth Goodman, CRO at Boost Payment Solutions, argues: “To view AI as primarily a cost saver is missing the point entirely.” The organizations seeing the greatest value are using AI to increase productivity, accelerate decision-making and unlock entirely new capabilities.

With industry leaders already benefiting from AI’s productivity gains, CIOs have no time to waste. Fixing the foundations should be the key first step for IT leaders looking to get ready for AI.

AI-powered enterprises are being built today. It’s time to get real about your AI readiness. Discover how to evolve your network for the next era in Tata Communications latest whitepaper.