New data centers are being announced. GPU capacity is being secured years in advance. Power has become a strategic resource. Enterprises are deciding where models should run and how quickly they can move AI from experimentation into production. Much less attention is being paid to what sits between the AI application and all that compute. That middle is getting complicated.
Networks, clouds, storage systems, security controls, APIs, data pipelines, edge infrastructure and existing enterprise applications all have to work together. Most organizations didn’t build these environments at once. They accumulated them over years, often through different technology cycles, vendors, acquisitions and business requirements.
AI is now asking all of it to behave like one system. I’ve started thinking of this increasingly complicated middle layer as the Muddle. It isn’t a new piece of technology or another layer to add to an architecture diagram. It’s a practical way of describing an infrastructure problem I see becoming much more important as AI moves into production.
We have powerful intelligence at one end and extraordinary compute at the other. Between them sits everything required to move, secure and deliver the data that makes AI useful. That middle may turn out to be one of the harder parts to get right.
Compute Is Only Part of the Equation
Much of the AI infrastructure conversation has centered on GPUs, data center capacity, power and cooling. That’s understandable. The scale of investment required to support AI is enormous. But compute capacity alone doesn’t determine AI performance.
At enterprise scale, data has to travel between the places where it is created, stored, processed and consumed. An AI workload might involve a corporate data center, a public cloud, a SaaS platform and an edge location before a useful result ever reaches an employee or customer. Every transition introduces another dependency.
The network has to perform. Security policies have to follow the traffic. Applications have to communicate correctly. Data has to be available where the model needs it. The infrastructure also has to provide enough visibility for IT teams to understand what is happening when performance starts to deteriorate. The GPU at the end of that chain can be extraordinarily fast. It still can’t process data it hasn’t received. That is what makes the middle so important.
AI Is Running Into Infrastructure Built for a Different Era
Enterprise infrastructure rarely starts with a clean sheet of paper. A large organization might have workloads spread across several clouds, applications delivered through SaaS providers, systems still operating in private data centers and hundreds of physical locations using different access technologies.
There may be multiple network providers involved. Security platforms may have been introduced at different times. Some locations may have fiber. Others may rely on broadband, wireless connectivity or a combination of technologies. None of this is unusual.
For years, businesses have been able to operate with a certain amount of fragmentation because many applications could tolerate it. Employees accessed applications, records moved between systems and transactions crossed networks without every component needing to operate as part of a continuous real-time process. AI changes the pressure on that environment.
Inference is increasingly distributed. AI applications can depend on information from multiple systems and locations. Agentic systems can create chains of interactions in which one request leads to another system, another model or another data source.
A delay that once affected a single application session can now become part of a much longer automated process. The infrastructure hasn’t suddenly stopped working. We’re simply asking much more of it.
The Muddle Extends Beyond the Data Center
There is another reason I think this issue deserves more attention: enterprise AI won’t remain confined to centralized computing environments. Think about where enterprise data is actually created. It is generated at hospital facilities, manufacturing plants, retail locations, warehouses, bank branches, offices, cameras, sensors and connected equipment. In many cases, moving all of that information to a distant location before acting on it isn’t practical.
Some AI workloads will run centrally. Others will move closer to the edge. Many will use both. That changes the infrastructure equation. Latency becomes an application consideration. Last-mile reliability can affect an AI workflow. Traffic routing can influence performance. Resiliency matters because an intelligent application isn’t particularly useful if the location generating the data can’t reliably reach it.
For infrastructure teams, this creates a much broader challenge than simply connecting a server to a GPU cluster. They have to think about the path from the point where data originates all the way to the place where intelligence is applied.
More Technology Doesn’t Necessarily Fix Complexity
Enterprise technology has a habit of solving one problem while creating another. A business adds another cloud platform because it provides a capability the organization needs. A new security product addresses a specific risk. Another connectivity provider improves coverage in certain markets. A monitoring platform gives one team better visibility.
Each decision can be perfectly reasonable on its own. Over time, though, someone has to operate the combined environment. That is where the Muddle grows.
AI can expose those seams because AI workflows frequently cross them. An application doesn’t care that networking belongs to one team, cloud infrastructure to another, security to a third and the data platform to someone else. It simply depends on all of them working together.
When performance falls short, finding the source can become difficult.
- Was the model slow?
- Was the compute environment constrained?
- Was there latency between locations?
- Was a security control adding delay?
- Was the required data unavailable?
- Was there congestion somewhere along the path?
Those aren’t theoretical questions. They are operational questions, and organizations will face more of them as AI becomes embedded in everyday business processes.
The Network Is Becoming Part of the AI Architecture
For much of enterprise IT history, connectivity was something applications used. AI is changing that relationship. Where a model runs, where its data resides and how quickly information can travel between the two can directly affect the experience an application delivers. That makes network design increasingly relevant to AI design.
The same applies to resilience. An AI application supporting one location may be able to tolerate a brief connectivity issue. An automated system operating across hundreds or thousands of sites creates a very different risk profile. This is why I expect enterprises to think less about connectivity as a collection of circuits and more about how the entire environment performs.
That means greater attention to path diversity, intelligent routing, redundancy, monitoring and the ability to identify and remediate problems quickly. The goal isn’t a more complicated network. It’s the opposite. The underlying infrastructure may become more sophisticated, but operating it has to become simpler.
Simplifying the Middle
There is a tendency in technology to assume the answer to complexity is another layer of technology. I think the more important opportunity is orchestration. Enterprises need better visibility across infrastructure that has historically been managed in separate domains. They need to understand application performance across cloud, network and edge environments instead of looking at each one independently.
Automation will become increasingly important here. Infrastructure can already respond dynamically to changing network conditions. Systems can identify anomalies and reroute traffic. Equipment can be monitored remotely and, in some cases, remediated without waiting for a technician to arrive on site.
AI will make those capabilities more sophisticated. Over time, I expect infrastructure to become better at making routine operational decisions on its own: selecting paths, identifying emerging problems, shifting workloads and responding to failures before they become larger service issues. There is an interesting irony in that. AI is exposing the complexity of modern infrastructure, but AI may also become one of the tools we use to manage that complexity.
The Next Infrastructure Problem
We are going to keep building bigger AI systems. There will be more compute, more models and more AI operating at the edge. Enterprises will find new places to apply intelligence, and the amount of data moving between systems will continue to grow. That makes the architecture between data and compute increasingly important.
The companies that handle this well won’t necessarily be the ones that accumulate the most technology. They will be the ones that make their technology work together. That requires treating networks, cloud environments, security, edge infrastructure and compute as parts of the same operational system.Â
For years, the technology industry has focused on making the endpoints more powerful. Now we need to pay much closer attention to what happens between them. Because that is where AI infrastructure is starting to get stuck. That’s the Muddle.
