Headline: AI’s Real Bottleneck Is Data Movement, Not Models

Key facts:

• Enterprises have invested heavily in AI models, compute, licenses, and GPUs, yet many cannot reliably access the systems they built.
• 59% of organizations report bandwidth issues, up from 43% a year earlier; latency challenges rose from 32% to 53%.
• A typical AI agent can generate up to 450% more total network traffic per task than a human doing equivalent work.
• Compute and GPU availability are rarely the primary bottleneck; congestion, routing, packet loss, jitter, and security inspection often matter more.
• Successful pilots often fail at global scale because networks were designed for predictable human transactions, not bursty machine-speed AI traffic.
• Leaders should assess end-to-end latency, data sovereignty, network and AI team alignment, and performance in the hardest geography before spending more.

The spending boom meets the network wall

For the past two years, enterprise technology budgets have been open wide. Organizations have poured billions into AI models, compute capacity, software licenses, and GPUs. The assumption was simple: buy the best models and enough processing power, and the AI transformation will follow. Yet despite those massive investments, a growing number of organizations are running into the same frustrating problem. They cannot reliably access what they just built.

When expensive AI workflows collapse, it is easy for leaders to assume the models are not working. The instinct is to blame the algorithm, the vendor, or the cloud platform. But the blame often lies elsewhere. Leaders consistently underestimate the connectivity requirements beneath their AI strategy. They treat the network as plumbing rather than as a core part of the AI performance equation.

Recent research points to a widening gap. 59% of organizations now report bandwidth issues, up from 43% just a year ago. Latency challenges have nearly doubled, surging from 32% to 53%. Enterprises are paying for that gap in performance, in user experience, and often in unnecessary spend aimed at the wrong part of the stack. More compute will not fix a request that cannot move quickly and reliably from a user to a model, through security inspection, across a cloud region, and back again.

Data movement, not compute, is the hidden bottleneck

An AI conversation from two years ago and an AI conversation today would be practically unrecognizable from each other. The market has moved from asking whether the technology is real to declaring full commitment and demanding that it work. For most of that shift, the dialogue has understandably centered on the structure of the models. But a model is only as effective as the infrastructure connecting it to users, applications, agents, and the other models on which it depends.

Enterprise AI is increasingly bogged down by data movement problems. Traditional enterprise networks were built around a predictable human transaction. Someone searches for something, the system fetches it, and the transaction ends. AI breaks that pattern entirely. One statistic remains striking every time it is said aloud: a typical AI agent can generate up to 450% more total network traffic per task than a human performing the equivalent work.

That is not just more traffic. It is a different kind of traffic. It is distributed, bursty, concurrent, and generated at machine speed rather than human speed. A successful AI pilot may prove that the model works, but it does not prove that the enterprise can run that model at scale. The pilot may sit in a favorable environment with close cloud proximity, ample bandwidth, and modest usage. Production is far less forgiving.

Why AI traffic behaves differently

Many enterprise networks were designed for employees who open a document, send an email, join a video call, or query a database. Those activities create traffic patterns that are relatively predictable. AI changes the shape of the load. An agent may call a model, retrieve enterprise data, invoke an API to another system, and trigger another model in turn. Each step introduces delay, and the cumulative delay across that chain is not easily predicted.

This is why examining only the model is misleading. The performance of an AI application depends on whether a request can traverse the full chain from user to network to security to execution and back without delay compounding at every hop. It depends on where data sits, how it is routed, how security inspection is performed, and whether local internet conditions introduce packet loss or congestion. The model may be excellent while the overall experience remains poor.

This pattern plays out every day across large enterprises rolling out AI strategy. In the initial environment, the deployment performs exactly as expected. Users are close to the cloud region, bandwidth is ample, and usage is still modest because the organization is not yet fully using the tool. Everyone declares success. Then the rollout goes global. Employees in Latin America, parts of Asia, or Africa suddenly see dramatically slower responses. The instinctive conclusion for many leaders is that the AI platform itself is underperforming, so the logical response is to spend more on compute, licenses, and capital expenditure aimed at expanding the platform.

The pilot-to-global rollout trap

That can be the wrong lever. When the full end-to-end transaction is examined, the model usually is not the problem. Consider the extraordinary scale of AI traffic as it moves from a branch office to a centralized security stack, then to a cloud region, then to a data-hosted backend somewhere else. Add inconsistent local internet performance, packet loss, and congestion, and the AI experience deteriorates. This is not because the model got worse, but because everything underneath it was never built to carry this kind of load across this kind of distance.

The challenge also varies widely depending on where in the world the organization is standing. At one AI symposium in Africa, attendees might have been expected to discuss which model to use, but the conversation instead focused on how to actually deploy AI. In Latin America, the discussion heard most often from customers is not about raw bandwidth but about latency, specifically because of how the region’s infrastructure routes traffic back to major cloud regions. Silicon Valley and São Paulo are not the same infrastructure conversation, and treating them as though they were is exactly how well-intentioned rollouts stall.

When an AI application performs poorly, the instinct is to look up the stack. Leaders ask whether the model needs fine-tuning, whether the vendor is reliable, or whether the cloud region is congested. But the answer is often underneath the application. It sits in the network path, the security architecture, the routing policy, and the geography of users, data, and compute.

Bandwidth is volume; latency is time

Bandwidth and latency may get treated as the same problem, but they are not. Bandwidth is volume, and AI increases it because organizations are moving richer data such as documents, images, video, and telemetry. Latency is time, and it is where the harder problem lives. A network can have plenty of capacity and still deliver a poor AI experience if requests take too long to travel across regions, security stacks, and application dependencies.

AI workflows are chained. An agent calls a model, retrieves enterprise data, invokes an API to another system, and that system may call another model in turn. The cumulative delay across that chain is not predictable, because no one knows in advance what will be asked or how far the request must travel. The user experience is shaped by the slowest or most congested hop, not by the average performance of the network.

That is why simply buying a bigger circuit does not solve the problem. Compute or GPU availability are not typically the bottleneck. What matters are congestion routing, packet loss, jitter, and where security inspection happens along the path. A larger pipe may help with volume, but it will not fix a route that sends traffic halfway around the world before it reaches the model or a security check that adds delay to every agent call.

Why bigger circuits do not solve the problem

Even hyperscaler infrastructure does not eliminate this challenge. The system’s speed depends on whether a request can traverse the full chain from user to network to security to execution and back without the delay compounding at every hop. When customers are certain that a major cloud provider must be the bottleneck, the infrastructure on their end is rarely the issue. If anything, the infrastructure is usually excellent, because that is what it is built to deliver.

The real question is whether a request can survive the full round trip. It must travel from the user, through the network, through security inspection, out to the cloud or data center, and back again without losing time at every handoff. As compute continues to improve, the path to accessing it becomes an increasingly important part of the performance equation. Enterprises risk spending billions on a Formula 1 engine and then trying to race it on a road never designed for that speed. They will not get the performance they paid for, not because the engine is wrong, but because the infrastructure underneath it cannot support what they are asking it to do.

Questions leaders should answer before spending more

There is no one-size-fits-all fix, but there are questions every leader should be able to answer before scaling an AI deployment. The answers reveal whether the organization is investing in the right layer of the stack or simply adding capacity to a path that cannot deliver it.

• Where do your AI users, models, and data sit: locally, regionally, or centrally?
• What is the data sovereignty picture, and can traffic legally and technically move across the regions your rollout depends on?
• Are you assessing the actual end-to-end network conditions experienced by AI users, including latency, jitter, packet loss, and routing, or relying on average network metrics that can conceal problems along specific routes or in particular locations?
• Do the team running your AI strategy and the team running your network infrastructure talk to each other, or are they operating in separate silos?
• And critically, what does performance look like in your hardest geography, not your best one?

Until those questions can be answered, organizations will struggle to scale, regardless of how well the pilot went. The risk is not only wasted budget. It is a loss of confidence in AI itself, even when the model is performing exactly as designed. Teams may conclude that the technology is not ready, when the real issue is that the data movement path around it was never engineered for machine-speed, distributed, bursty workloads.

AI strategy and network strategy can no longer be separate conversations. They must be elements of the same conversation, from the very beginning. The enterprises that succeed will not simply buy the most powerful models or the largest GPU clusters. They will understand where their users, data, and models sit, how requests travel between them, and what must change in the network and security architecture to make the entire chain perform. The model may be the engine, but data movement is the road, and no amount of engine horsepower can compensate for a road that was never built for the race.

Source:Network World News