Tech users face a growing conundrum: the laptops and other devices we use can no longer run frontier workloads such as large language models (LLMs).
So, for years that has left us with a binary choice. We either carry a heavy machine locally that can still handle only entry-level AI work, or send the work to a GPU in a data center.
The latter choice is now creating more and more tech giants that grow by any means necessary. They’re building nuclear plants for mega data centers, training ultra-large AI models, and retaining all user data.Users pay for every API call and every token. Every request travels over the internet, so data isn’t nearly as secure, and it waits in a queue, competing with untold numbers of other requests. That’s the only solution now – if you want to use the frontier AI models, you have to send the data out somewhere.
In trying to solve this conundrum, our starting point at WiCi was simple: we wanted to bring frontier computing to everyday devices, for everyone. But the weight and power consumption are hard limits – we therefore separate the GPU from the device. Put the computing somewhere with a reliable power supply, and let every nearby device use it, the same way they all share one Wi-Fi connection.
Wi-Fi 7 gives us a strong wireless foundation, but making GPU workloads practical over that connection still requires substantial system design. Even the best Wi-Fi remains far slower than the PCIe connection inside a desktop.
At the same time, the frontier AI models, with trillions of parameters, are far beyond the capacity of even the best consumer card. The RTX 5090 only has 32GB memory, while frontier models can require terabytes.
Our work rests on two pillars. The first is wireless GPU computing: we optimize the network stack, firmware, and protocols to make full use of available wireless capacity, and rethink GPU communication to eliminate unnecessary transfers and round trips. The second is storage-backed inference: streaming model weights from SSD into GPU memory so a single consumer GPU can run models far larger than its VRAM.
The Latency Issue
My academic work as a professor has focused on computer networks and computer architecture. 80% of our team are PhDs who have worked on wireless protocols and AI systems for the past decade, publishing over 60 academic papers. My lab’s research on real-time streaming has been deployed commercially in Tencent, Alibaba, and many other cloud companies, where players feel every millisecond of delay.
The biggest lesson from that work is that you can’t win by pushing more bits through the network. You win by rethinking what has to cross it at all.
In practice, we solved a series of problems with the following approach:
- First: Push Wi-Fi to its capacity and make it consistently responsive. A fast wireless link can still stall when packets pile up in queues or repeated retries hold up other traffic. We redesigned how packets are queued and how transmission rates and retries are managed, so these decisions account for signal quality, queued traffic, and how long packets have already waited. This helps prevent brief changes in wireless conditions from turning into long pauses, keeping latency more predictable while maintaining high throughput.
- Second: Send less over the air. Even at capacity, Wi-Fi is far slower than PCIe, so we built our own caching system to remove unnecessary transfers and waits. Traditional GPU software assumes communication is cheap. It may repeatedly ask for an unchanged device property or send data the GPU already has. Over wireless, those patterns create extra traffic and delays.
- Third: Change how commands are processed. Answer repeated queries locally when the answers remain valid, avoid resending duplicate data, and keep reusable state near the GPU. We also let independent commands proceed without waiting for a reply after each one. The difficult part is identifying which waits can safely disappear while preserving every dependency the application actually needs.
- Fourth: redesign the full stack around the SSD to boost the performance. SSD inference engine, file systems, drivers, → the fastest framework to run AI models from SSD.
The Result
With these technologies, an entry-level laptop can use WiCi One’s RTX 5090 edition to run a 26-billion-parameter model at over 500 tokens per second, or even the latest DeepSeek model at 10 tokens per second (to a typical reading pace), without any quantization or compression – at original quality.
To put that scale in perspective, one commercial cloud provider deploys a trillion-parameter model across eight data-center GPUs with more than two terabytes of combined GPU memory. WiCi makes it accessible on your own network with a single consumer GPU.
WiCi streams weights from SSD into GPU memory as needed. Our optimization work focuses on retaining reusable weights, fetching others ahead of use, and organizing reads so the SSD operates efficiently. This lets us run models larger than the GPU’s physical memory, while we continue improving their speed. Models that fit entirely in VRAM remain fastest.
Today’s frontier AI models are no longer measured in billions of parameters, but in trillions. Running a one-trillion-parameter model entirely in GPU memory requires roughly a terabyte of VRAM. That’s millions of dollars of hardware, about the price of an apartment in a major city. Even a high-end desktop like a Mac Studio tops out at models of around 100 billion parameters, which are capable but not frontier.
The WiCi One treats a fast PCIe Gen 5 SSD and the GPU’s VRAM as one unified memory pool, while a redesigned file system allows model weights to be streamed to the GPU as they’re needed. Models that fit in VRAM (the RTX 5090 in the WiCi One has 32GB of VRAM) will always run fastest, and streaming the very largest models is slower and something we’re still optimizing. But the alternative today is not being able to run them locally at all. People shouldn’t have to choose between frontier-level quality and keeping their data private.
We’ve deliberately tested the WiCi One in public settings like coffee shops, rather than a quiet lab, because coffee shop Wi-Fi is notoriously bad. In some of those locations, there are more than 20 competing Wi-Fi networks. Because our router board operates across different frequency bands and channels, it coexists with existing networks rather than interfering with them.
It also behaves like Wi-Fi in other ways. One unit comfortably serves five to ten devices. If you need more, you do what you’d do with Wi-Fi: add a few units and mesh them together. And because the device is no longer powering its own GPU, the trade-off on battery life is small. The Wi-Fi radio uses some extra power, but far less than carrying a high-end graphics card.
Our breakthrough has massive implications for robotics, wearables, and AI-native devices.
Everyone understands what a laptop could do with an RTX 5090. But nobody has ever imagined a smartphone or a pair of smart glasses with one. Those devices don’t even have the drivers for it.
My rule of thumb: any device that can’t support a 1,000-watt power supply can benefit from this approach.
What’s Coming Next
Physical AI labs are now building robot models that have outgrown on-robot processors. The cloud is where those models are trained, but a robot making split-second decisions needs inference close by. One full-size GPU on site, a single Wi-Fi hop away from every robot in the building, handles that real-time work, while the cloud keeps doing what it does best. And the camera and sensor data from a home or factory can stay on the premises.
The evolution may echo how internet access changed over the past two decades. First, you plugged in with a cable. Then Wi-Fi arrived, and we started connecting wherever we went. Today, we’re connected over cellular almost all the time.
Computing can now begin to follow the same path.
Right now, GPU power is locked inside a box or a data center. The next step is compute you connect to, the way you connect to Wi-Fi. A household buys one unit, and every device in the home can use it from any room. A café buys one, and every customer gets access. After that, the natural next step is cellular, so that compute follows you everywhere, the same way the internet does today.
That’s why we’ve made our SDK and our inference engine public – we aim to open everything that can benefit the community.
The GPU has spent a quarter century tethered to a box. The next generation of AI devices won’t need to carry their compute at all, which will be a game-changer. Nobody yet knows what a phone could do when it has a 5090 behind it.
Once this technology goes mainstream, it will be thrilling to see how developers answer that question.
About the Author
Bio: Zili Meng is the Co-Founder and CEO of WiCi, where he leads the technical strategy behind the world’s first wireless local compute layer for on-site AI. An Assistant Professor at HKUST and head of the SPARK Lab, Zili brings deep expertise in networked systems, machine learning, and computer networks, with award-winning research recognized by ACM SIGCOMM, IRTF, and USENIX NSDI. His technology stack innovations have been deployed at scale across major enterprise platforms, including Tencent Cloud Gaming.
