Internal connectivity is becoming an integral part of AI strategy. As AI data centers scale with high-performance XPUs to support large language model training and inferencing for agents and applications like ChatGPT, Gemini and Copilot, how those XPUs connect matters more than ever. The network inside the data center shapes the performance and scalability of distributed processing, the effectiveness of model training and use, the efficiency of compute resources, and how quickly organizations can turn AI infrastructure into business value. Intra-data center connectivity is now a strategic decision rather than a purely technical one.
As with so many things in life, designing connectivity inside modern AI data centers is an exercise in compromise. Each architectural decision is a balance between competing priorities – power, cost, performance, flexibility, ease of deployment and scale, serviceability.
As systems scale to support ever-larger AI workloads, it’s clear no single approach can optimize all dimensions. If a system can’t have it all, the question becomes: what matters most in the environment and for an organization’s business and operational goals?
Key metrics for AI intra-data center connectivity
Understanding the key metrics for intra-data center connectivity and their possible trade-offs helps frame the discussion around operational goals rather than technology preferences.
Performance – capacity and latency
The network’s ability to move large volumes of data quickly and with integrity between compute resources, while minimizing delays that can slow or jeopardize distributed AI workloads.
Power
The energy required to operate the connectivity infrastructure. Power not only impacts operational cost but also drives cooling requirements and can affect overall data center scalability. In power-constrained environments, each watt saved on connectivity can be reallocated to XPUs and directly monetized.
Cost
The total financial impact of connectivity, including up-front investment, ongoing operational expenses and long-term gains or penalties as the system scales. This cost is not passed on to the AI user, who only pays for compute.
Flexibility
The ease with which the infrastructure can adapt to changing needs, including evolving traffic patterns and multi-vendor environments.
Ease of deployment and scale
How quickly and simply connectivity can be deployed and integrated, which affects time-to-market, roll-out complexity, and the need for specialized skills or heavy processes.
Serviceability
The ability to maintain, repair and upgrade the system with minimal disruption, influencing downtime, operational risk and long-term manageability.
Pluggable versus integrated optics: two design paths
Once key decision metrics are outlined, the discussion shifts to the architectural options. Each option has a different balance between performance, power, flexibility, cost and operational considerations.
In AI intra-data center connectivity, it is useful to distinguish between scale-up and scale-out networks. Scale-up connects XPUs within a rack or pod. Copper still dominates very short-reach scale-up connections, although its limits are becoming more evident as bandwidth increases. By contrast, scale-out connects top-of-rack switches across racks and rows to form a distributed compute fabric.
Optical connectivity is essential in scale-out networks, where speeds have progressed from 100 Gbps to 400 Gbps, 800 Gbps, and now to 1.6 Tbps. Analysts such as LightCounting expect sales of Ethernet optical transceivers and co-packaged optics to double within five years, reaching hundreds of millions of units annually. Intra-data center demand from expanding AI clusters is the main driver.
Optical connectivity architectures fall into two broad categories: modular, pluggable approaches and more integrated designs such as near‑packaged and co‑packaged optics.
Pluggable optics: In traditional pluggable designs, connectivity sits at the edge of the shelf, separate from the core processing components. Pluggable optics are the most widely deployed connectivity solution inside the data center. This approach supports flexibility, ease of deployment, and straightforward serviceability by allowing components to be upgraded or replaced independently, as needed. Itsimplifies operations and reduces risk while enabling multi‑vendor environments through industry standardization.
Integrated optics: Newer approaches move connectivity closer to the switching fabric. More integrated designs such as near-packaged optics (NPO) and co-packaged optics (CPO) shorten the electrical path between the switch ASIC and the optics, reducing losses and inefficiencies. By bringing switching and connectivity closer together, these designs can improve efficiency and lower power consumption, which is increasingly important as systems scale. They also enable higher connectivity density than pluggables that sit on a faceplate.
However, NPO and CPO change how optics and switches are designed and built, and limit the flexibility of modular approaches.
Not all pluggables are created equally
There are various signal processing architectures within pluggable optics, often referred to as fully retimed optics (FRO), linear receive optics (LRO) and linear pluggable optics (LPO).
FRO are the most mature and well-established and include in-module signal retiming and processing for both transmit and receive. They provide very robust performance and are easy to deploy, working across a wide range of environments and with broad support. This comes at the expense of higher power consumption, as signal processing is performed locally in the module.
LRO, also known as half-retimed optics (HRO), implements in-module signal retiming for transmit while using a linear, non-retimed receiver. They rely on signal processing in the switch ASIC in the receive direction, reducing overall power and latency compared to FRO-based solutions while maintaining flexibility and ease of use.
LPO pluggable modules fully remove signal processing from the optical module and rely on the switch ASIC to manage all signal integrity. LPO further improves power efficiency and reduces latency, butrequires LPO-compatible switches and has shorter optical reach. Their dependency on the host means end-to-end system validation is required, which can introduce additional operational complexity.
FRO, LRO and LPO represent different signal processing architectures within pluggable optics. By contrast, extra‑dense pluggable optics (XPO) is an emerging approach along an orthogonal dimension, redefining the module form factor itself. It is designed to deliver much higher bandwidth density and improved thermal efficiency compared to established pluggable form factors such as OSFP.
XPO extends the pluggable model by using a liquid-cooled form factor that fits up to 12.8 Tbps per module. The goal is to increase front‑panel density while preserving the operational model of pluggableoptics, including modularity, serviceability and ease of deployment, upgrade and scaling.
Bringing optics on board
In more integrated optics‑switch designs, near‑packaged optics refers to an approach where the optical interfaces are moved much closer to the switch ASIC: inside the physical switch but still separate from the ASIC silicon. Co‑packaged optics takes this integration further by embedding the optical interfaces directly within the same package as the switch ASIC. In NPO and CPO architectures, external light sources are often used to keep lasers outside the switch enclosure, improving thermal management, reliability and serviceability, and isolating the system from potential laser failures.
NPO and CPO designs improve power efficiency and signal integrity and can support larger‑scale systems, but they are initially more complex and challenging to service due to tighter componentcoupling. These architectures typically require close co‑design between optics and switch or ASIC vendors, particularly in the case of CPO, often leading to highly proprietary solutions. While this article describes on-board optics in the context of switch‑based, scale‑out networks, the same concepts can extend to XPU ASICs in emerging tightly coupled scale‑up architectures.
Figure: Intra-data center architectural options: pluggable optics versus integrated optics
Coexistence, not convergence
A key implication of the trade-offs discussed is that no single solution is likely to dominate AI data center connectivity. Multiple options will coexist, each suited to different use cases and priorities, whichreflects the diversity of approaches to AI scaling.
Some environments will prioritize modularity, multi-vendor compatibility, flexibility and serviceability, leaning toward approaches that are easier to operate and evolve, even if they are less efficient at the extreme. Others will prioritize performance and efficiency at scale, potentially using tighter integrated environments for the largest AI data centers where every watt and every microsecond counts.
This coexistence also reflects organizational differences. Enterprises, cloud providers and specialized AI operators may have very different priorities in terms of cost structure, time-to-market and operational models. Connectivity decisions will increasingly reflect these strategic differences rather than a universal industry direction.
Connectivity as a strategic choice
Instead of replacing existing approaches, new connectivity technologies are adding to the toolkit. This creates more options for designing AI infrastructure, where connectivity is shaped by context rather than consensus. Pluggable optics continue to improve in speed, density and efficiency. But connectivity is also moving closer to computing and switching silicon, changing how AI systems are designed, deployed and operated.
The challenge is not choosing a winner architecture but selecting the right fit for a given scenario. Success will depend on making informed trade-offs that align with business goals and operational realities. In that sense, intra-data center connectivity is no longer just a technical layer — it is a strategic lever.
