Asari AI’s co-inventor agents got up to 32% more performance out of a popular AI inference engine, and improved themselves to get faster at it. Zihao Li, the company’s head of infrastructure, on the advances in AI that are needed to make this happen.
This summer, Asari AI’s “co-inventor” agents used the scientific method to optimize vLLM, the open-source engine widely used to serve large language models. This made multiple frontier large language models (LLMs) with up to trillions of parameters faster and more responsive by up to 32% running on NVIDIA B200 GPU accelerators. LLMs such as DeepSeek v4 Pro, GLM 5.2, Kimi K3, and Thinking Machines’ Inkling all saw double-digit improvements in performance across multiple load levels within days.
Such gains are not easy to get. vLLM is a highly dynamic and complex codebase. A large open-source community contributes to it, and it has to be retuned for every new model — work that calls for engineers fluent in model architecture and GPU memory behavior at once. Few engineers are fluent in both, and improvements are not sitting on the surface.
Finding and verifying this kind of gain requires many advances in AI infrastructure. This is where Zihao Li comes in. As Asari AI’s head of infrastructure, Li built critical parts of the software that powers the co-inventors and the environments they work in.
“Infrastructure for modern AI agents needs to get many things right: make building agents easy, make experiments fast, and make the measurements worth trusting. We built a modular framework that makes this happen.” explains Li.
The co-invention process is implemented using a general-purpose multi-agent framework that can be used for many problems. Earlier this year, a previous generation of agents used this framework to translate four production C libraries into Safe Rust. One of those was a high-precision mathematics library built to run in space; a verifier of European Space Agency flight software confirmed the translation passed the full suite of ESA-standard tests.
Li elaborates: “It is critical to make sure that the solutions found by our agent are trustworthy. That’s why we’ve invested a lot of effort into various verification mechanisms to make sure our answers are correct.”
The same framework now serves the inference optimization work and other applications, keeping verification as a key feature and making sure agents can’t cheat. Every code change is checked independently from the agent that proposed it. Every model’s output is checked statistically to make sure it hasn’t drifted too far from the original.
Another part of automating the scientific method is conducting controlled experiments. Li’s infrastructure lets the agents run carefully controlled performance evaluation experiments in an isolated “sandbox” environment. This means that every sandbox should be identical to the other
and should contain all the knobs that the agents might want to tune. In practice, this covers the model, the inference software, but also the operating system itself.
“Optimizing inference is a system problem, not just a code problem,” Li says. “The agents have to reason across the whole software and hardware stack, so we need to make sure the environment itself is stable and all information that the agent needs is there, every time.”
Because the company often runs many optimization attempts and experiments in parallel, and repeatedly, the connection between agent and sandbox needs to be reliable, secure, and support long-running jobs. To make this happen, Li’s infrastructure uses private networks that support reliable and secure communication and data transfer across remote sandbox machines and the agents running in cloud clusters.
And this reliable distributed system should also be fast and efficient. Speed is a notorious bottleneck, as building inference software can take up to an hour. This is why the infrastructure also features dynamic scheduling of GPU resources and managing persistent data and caches to avoid redundant code compilation and execution, across experiments.
What excites Li in particular, is that the infrastructure lets the agents get better over time. In one example, agents learned to avoid a distributed deadlock that had cost 44 minutes of waiting by a previous agent.
Li: “An agent that works something out does not just keep it. It shares that insight with other agents and lets the group find out what works and is worth keeping. We’ve set up our infrastructure to let the agents keep a record of when something was learned and why. What the agents work out on one model can become an insight for the next.”
Because agents run for days and many run at once, they generate an enormous amount of information. The company’s infrastructure controls both the logic and reasoning about useful knowledge, but also implements a storage, organization, and redistribution process to efficiently share information between agents.
“We designed a centralized registry where each agent can upload their insights after their work, and other agents can download and make use of what other agents have learned.” explains Li.
“It’s not that an agent got a good result once,” Li says. “It’s that the next one can reuse what the last one found out. We believe that our infrastructure design and agent engineering sets the stage for many new discoveries, and we’re just getting started.”
