Why Broadcom Teamed Up with Rivals to Fix AI’s Billion-Dollar Networking Problem

Something unusual happened in the AI industry, and almost nobody noticed. Broadcom, Nvidia, AMD, Microsoft and OpenAI — companies that compete directly for silicon contracts, cloud workloads and the market’s attention — spent two and a half years jointly developing a networking protocol called Multipath Reliable Connection, or MRC. By the time they announced it, the protocol was already running inside OpenAI’s largest supercomputers, including the Stargate facility in Abilene, Texas, and Microsoft’s Fairwater sites.

Competitors often collaborate only when a shared problem gets expensive enough. This one came with a number: in 2022, Meta publicly disclosed that its GPUs sat idle up to 57% of the time — as the network connecting them couldn’t feed them data fast enough.

Hasan Siraj, vice president of products for Broadcom’s Core Switching Group, says hyperscalers began complaining years ago that “the network is getting in the way.” Cloud companies have been building large-scale infrastructure for decades. So, what changed? AI training.

“Imagine spending billions of dollars on an asset you can’t fully use,” Siraj says. AI, in his view, is fundamentally a distributed computing problem. Modern models are too large to train on a handful of chips, requiring hundreds of thousands of XPUs to function as a single system. “The network is the glue that binds them,” he says. “That’s why we say the network is the computer — the network is the supercomputer.”

The industry has spent three years measuring AI supremacy in GPU counts. Siraj contends the count matters less than the connection; a hundred thousand chips that can’t talk to each other efficiently behave like a much smaller, much more expensive machine.

Why a 20-Year-Old Protocol Was Quietly Failing

Most AI training clusters rely on RDMA (remote direct memory access), a protocol that lets one server read and write another server’s memory directly without routing data through the operating system. But RDMA is more than two decades old, built for an era when large computing jobs spanned dozens of machines, not hundreds of thousands.

At AI scale, those assumptions began to fail. “It did not support multipathing,” says Siraj. RDMA typically uses one route between two servers; if that path becomes congested, traffic waits. RDMA also expects data packets to arrive in sequence. “If packets arrived out of order, the destination would drop those packets,” Siraj says. 

The retransmission mechanism proved equally inefficient. RDMA relied on an older technique called Go-Back-N, where losing a single data packet forced the sender to retransmit not only that packet but every subsequent one — more traffic precisely when the network was already congested. “There was not a lot of telemetry,” Siraj says, leaving operators with limited insight into where traffic was slowing or why.

MRC modernizes all four. It can spread a single connection’s traffic across multiple network paths at once — up to 128 with Broadcom’s Thor Ultra network card. It accepts packets out of order and retransmits only the data that’s missing. It incorporates a technique called packet trimming to maximize bandwidth utilization: instead of discarding a packet during congestion, a switch removes its payload and forwards only the header, letting the receiving system identify and request exactly what’s missing.

The Alliance Behind AI’s Biggest Infrastructure Bet

Siraj notes that MRC worked because each participant saw a different piece of the problem.

“When you’re building such large clusters, it’s not one company,” he says. Networking vendors, GPU makers, cloud operators and AI labs each approached the challenge from their own perspective. Broadcom focused on the network, Nvidia on the GPUs, OpenAI on model training and Microsoft on operating massive deployments. “There was a lot of learning from people who had their domain expertise,” Siraj says. “That was the big revelation.”

The urgency is reflected in the growth of AI clusters. “We were talking about 4,000 GPUs, 8,000 GPUs—maybe 32,000,” Siraj says. “Now we have clusters in production that are 100,000 GPUs.” The next conversations, he adds, are about scaling from hundreds of thousands to as many as a million XPUs.

However, it’s interesting to note that Amazon Web Services (AWS)—the world’s largest cloud provider—is not part of MRC. AWS developed its own transport protocol years ago to solve many of the same networking challenges within its proprietary infrastructure. Moreover, Broadcom helped create a vendor-neutral standard that also happens to run on Broadcom hardware. 

Siraj doesn’t dispute the commercial incentive. Instead, he asserts customers demanded an open, multi-vendor networking standard for AI clusters at this scale, and Broadcom believes its products can compete on that foundation.

Why Eliminating a Third Network Tier Matters

Large AI clusters connect GPUs through layers of switches. As clusters grow, each additional tier adds hardware, latency and power consumption. MRC’s multi-plane design changes that equation. Instead of treating a network card’s full bandwidth as a single connection, it divides that bandwidth across several parallel networks. The result, Broadcom says, is that its 100-terabit Tomahawk 6 switch can connect up to 128,000 GPUs using two switching tiers instead of three.

Eliminating that third tier, according to Siraj, avoids roughly 67% more optical interconnects, cuts the switch count to a third, lowers hop latency by about 67%, and halves power consumption. The power savings may matter most, as electricity has become the limiting factor on how large AI data centers can grow. “Every watt of power that you save me allows me to put more compute in the data center,” Siraj says.

Networking still accounts for only about 15% to 20% of an AI cluster’s cost, with GPUs consuming most of the budget. Siraj calls that shortsighted. “If I can increase network performance by 10 or 15%, essentially the network has paid for itself.”

AI’s Biggest Clusters Are Choosing Ethernet Over InfiniBand

Five years ago, the presumed winner in AI networking was InfiniBand, the proprietary interconnect Nvidia acquired through Mellanox. Broadcom bet instead that Ethernet—the open standard that has powered data centers for decades—would prevail because no single vendor controls it.

“That debate has settled. If you look at anybody building the largest clusters across the hyperscalers, everybody is doing scale-out over Ethernet. Even Nvidia has a whole Spectrum offering around this,” Siraj says.

The shift is reflected in the Ultra Ethernet Consortium, the industry group formed in 2023 to modernize Ethernet for AI workloads by redesigning its congestion handling and transport behavior for massive GPU clusters. It now counts more than 175 member companies.

The case becomes even stronger beyond a single data center. AI clusters are running into a limit that has nothing to do with chips: power. A facility can draw only so much electricity from the grid, capping how many accelerators fit under one roof. The only way to keep scaling is to connect multiple data centers—sometimes hundreds of kilometers apart—and make them operate as a single system. “There is no other technology other than ethernet that can do this today,” Siraj says. “There’s nothing available.” 

The one frontier still dominated by proprietary technology is inside the rack—what the industry calls scale-up: the ultra-short connections linking GPUs just inches apart, where Nvidia’s NVLink remains dominant. These links carry the heaviest traffic in the system, and hyperscalers designing their own accelerators have resisted opening them. This is where they differentiate through custom load balancing, encryption and memory handling.

Even that wall is beginning to crack. The Ethernet for Scale-Up Networking working group—launched last October by 12 founding members, including Meta, Microsoft, Oracle and OpenAI, to extend Ethernet into the one domain it has never served—has since grown to nearly 200 members and published its first specification. If the effort succeeds, every layer of AI networking—from chip-to-chip communication inside a rack to links spanning entire data centers—could eventually be based on open standards rather than proprietary interconnects.

The deadlines make the stakes tangible. Because a stalled network leaves every GPU behind it waiting, hyperscalers now expect networking infrastructure to be deployed and validated months before the accelerators arrive. “The network should be ready three to six months before compute shows up,” Siraj says. “It cannot be the bottleneck.”

The Control Layer Verdict

The AI buildout just exposed what really controls AI. When Broadcom, Nvidia, OpenAI and Microsoft quietly spent two and a half years redesigning how AI clusters communicate, they answered a question the industry rarely asks: What actually limits intelligence at scale? It wasn’t the models or GPUs. It was a transport protocol built for a different era of computing. Rather than solve that problem alone, competitors chose to rebuild the foundation together. The decisions that shape AI aren’t always made by the companies with the biggest announcements. Rather, made deep in the networking stack by engineers whose names never make the headlines.

Author:

Victor Dey is a tech analyst and writer who covers AI, data science, startups, and cybersecurity. A former AI editor at VentureBeat, his work also appears in New York Observer, Fast Company, Entrepreneur Magazine, HackerNoon, and more. Victor has mentored student founders at accelerator programs at leading universities including the University of Oxford and the University of Southern California, and holds a Master's degree in data science and analytics.

Follow Victor Dey @iamvictordey