Why the right data movement strategy is defining the next era of AI compute
At the recent Chiplet Summit 2026, a panel titled “Best Way to Make Chiplets Work” brought together leaders from across the semiconductor ecosystem to discuss a defining challenge in advanced system design: how to ensure heterogeneous chiplet systems operate as a cohesive, energy-efficient whole.
The discussion frequently returned to emerging standards, such as Universal Chiplet Interconnect Express (UCIe) and other die-to-die interconnect technologies. These standards are essential for chiplets to communicate across advanced packaging substrates.
Yet the consistent message emerging from the conversation was that connectivity at the physical layer is necessary, but not sufficient. For AI systems, making chiplets truly work requires clarity at higher levels of system architecture and the next phase of chiplet innovation will depend less on packaging breakthroughs and more on the architectural intelligence governing how chiplets collaborate.
Moving bits is only the beginning
Standards like UCIe represent a critical step toward a modular semiconductor ecosystem. By defining a standardized die-to-die interconnect—including physical-layer and protocol-level mechanisms—they enable high-bandwidth communication between multiple dies within a single package, but moving bits between chiplets is only the first step.
While these standards provide essential connectivity and protocol foundations, they do not by themselves define the full system architecture, coherency model, workload behavior, or data-movement strategy required for heterogeneous AI systems. In systems combining CPUs, GPUs, AI accelerators, memory dies, and other specialized processors, shared communication semantics become essential.
As a result, system architects must determine how chiplets interact at a higher level, including:
- Which protocols govern communication
- Should interactions be coherent or non-coherent
- How devices negotiate capabilities
- How data consistency is maintained
Without these architectural agreements, the interconnect becomes little more than a transport mechanism and the real work of system integration occurs above that layer.
AI workloads expose the data-movement challenge
The rise of large-scale AI computing has amplified the importance of these architectural decisions.
For decades, improvements in semiconductor performance followed the trajectory of Moore’s Law, where increasing transistor density produced predictable gains in computing power. Today, however, AI workloads are scaling faster than transistor efficiency improvements alone can sustain and the energy cost of moving data is becoming as significant as the cost of processing it.
Architectural decisions play a critical role in determining power usage through system efficiency and no single communication model fits every use case. Effective architectures align communication mechanisms with workload semantics to optimize for both power and performance.
The need to shift system thinking earlier
Another recurring theme across the chiplet ecosystem is the need to “shift left” in system design, where early architectural planning is essential as chiplet architectures increase complexity dramatically. Previously, traditional system-on-chip development allowed many integration challenges to be addressed relatively late in the design cycle.
This shift left means that instead of designing a single integrated chip, architects must consider the interactions among multiple dies that may originate from different design teams, or even different companies.
Interoperability is still an ecosystem problem
Chiplets promise a future where semiconductor components can be mixed and matched more easily. But the broader ecosystem needed to support that vision is still developing and many heterogeneous chiplet systems are built by small groups of closely collaborating partners who coordinate specifications, verification flows and integration tools.
The long-term goal is more ambitious: an open marketplace where chiplets from different vendors can be combined with confidence. Achieving that goal requires solutions beyond physical connectivity and faces key challenges, including:
- Shared behavioral models across vendors
- Trusted verification IP
- Pre-silicon compliance testing
- Standardized system descriptions
Integration becomes the differentiator
The semiconductor industry is entering a new era driven by heterogeneous integration, and advanced packaging technologies, standardized die-to-die interfaces and chiplet marketplaces are expanding the design possibilities available to system architects. At the same time, AI workloads are placing unique demands on compute density and energy efficiency.
Combined, these trends are creating both opportunity and complexity.
Signal integrity, power delivery, thermal management and interoperability remain formidable engineering challenges and architectural decisions governing how data moves through the system may ultimately become the less visible but all-important competitive advantage.
AI and advanced computing demand heterogeneous integration, which in turn demands scalable, energy-efficient data movement, and scalable data movement requires deliberate system architecture.
Therefore, making chiplets work is not simply about connecting dies. It is about designing how those dies communicate, collaborate, and behave as a unified computing system. The organizations that are mastering that integration are already shaping the next generation of AI infrastructure.
