Cysic CEO Says Paying for an AI Model Doesn’t Prove Your Host Ran It

A company can use an open-source AI model without buying the hardware to run it. Third-party hosts do that work on their own servers and quote prices by the million tokens. For an engineering team choosing among hosts that list the same model, a lower rate looks like an opportunity to cut costs without changing the software it has already tested. The difficulty is knowing whether the cheaper host runs that software the same way.

Recently, a developer who sent 18 million messages through OpenRouter, a service that routes requests to AI hosts, found that the same named model scored 90% on a benchmark with one host and 75% with another. OpenRouter says it has no reason to believe any provider deliberately compromised quality. A host can reduce its computing costs by running the advertised model at lower numerical precision. It could also serve a smaller model while continuing to charge for the larger one. Either could produce an answer that looks reasonable to the employee who asked the question.

A buyer trying to establish which model produced that answer would need evidence from the computation itself.

Leo Fan, co-founder and CEO of Cysic, originally wanted to give buyers that evidence with every response. His company’s CyAI marketplace now audits a sample of requests and requires hosts to put up money they can lose if an audit catches them switching models.

“The approach we are taking with the product we launched several months ago represents a compromise. Attaching a certificate to every inference using zero-knowledge proofs remains impractical because of the cost,” Fan tells me. “Whether measured in processing time or capital expenditure, the expense is still prohibitive. Even today, generating a proof for every inference costs roughly 80 times as much as running the inference itself.”

How CyAI Audits AI Model Providers

Since Aug. 31, CyAI has let independent providers sell inference, the work a model does when it answers a request. Providers set their own prices, and a public blockchain records payments for the work. That record can settle a question about how much a provider earned. It takes a different kind of evidence to establish what the provider ran.

The marketplace chooses requests at random for inspection. Its audit design draws on Immaculate, an academic framework whose authors include Dawn Song and Jiaheng Zhang. A provider that saves a little money on each request by switching models has to weigh those savings against what it could lose if an audit catches it.

The provider has to make that calculation before it knows which responses will face inspection. “We use sampling to verify that users are receiving the models they requested. By combining zero-knowledge proofs with other game-theoretic approaches, we can identify providers attempting to deceive their customers,” Fan says. “The goal is to detect dishonest providers during the verification process and ensure they face consequences for their actions.”

The Hidden Costs of AI Model Verification

Fan explained that enterprises can verify a provider’s inference results by running the same input through a second machine and comparing the outputs. If the results fall within an acceptable range, the original output can be considered valid. 

However, this approach is prohibitively expensive because it requires another powerful machine to repeat the entire inference process, effectively duplicating the computational effort. “There are also significant limitations on how much you can verify. Given the computing resources required, you can only re-execute a very small portion of the inference results. Beyond the computational expense, there are privacy concerns as well. Re-executing the inference requires using the customer’s data again, which creates a risk of exposing that information.”

A zero-knowledge proof changes who bears that cost. Whoever generates the proof does the expensive work of establishing that the specified model produced a result. Someone else can check the proof on modest hardware without seeing the underlying private data.

“Once a proof has been generated, it can be verified and reverified at any time without having to repeat the original computation. You can simply download the proof along with the required parameters and independently perform the verification,” Fan says. “There is no need to execute the original inference again on another powerful machine. That makes the entire verification process considerably simpler and more efficient.”

Turning Idle Gaming GPUs Into AI Computing

After earning a doctorate in computer science at Cornell University and teaching at Rutgers University, Fan helped build Cysic around the idea that specialized chips could speed up proof generation for blockchains such as Ethereum. Conversations with potential buyers persuaded the company to invest in software that could also use off-the-shelf Nvidia graphics processors.

As demand for proofs from networks that help Ethereum process transactions declined, Cysic had machines it could put to work on AI. Its network includes the graphics cards people use for video games. That gives it access to computing power beyond the big cloud providers, but it leaves the company with a problem that cryptocurrency miners rarely faced. A contributor can offer a powerful machine at an attractive price and earn nothing if nobody needs an answer that afternoon.

“Previously, when you mined Bitcoin or Ethereum, you could hold on to the tokens and sell them later once their value increased. AI inference operates differently. You cannot simply store inference tokens and wait for their value to appreciate in the same way,” Fan says. “You either have a customer actively consuming those tokens, or the available computing capacity goes to waste. That is a fundamental difference between cryptocurrency mining and AI inference.”

Cysic supplies some demand through its own software, which lets AI agents work together using models on the network. Those jobs can keep a contributor’s machine occupied while the company attracts outside customers. For the same reason, a rising count of tokens served tells only part of the commercial story. Repeat purchases by outside companies would show whether buyers see enough value in the service to keep sending work its way.

GetDeploying’s index put the median hourly price of an Nvidia H100 at $4.19 at specialist GPU clouds and $7.89 at the largest cloud providers. Those figures compare data center chips. They show how much the host can affect the price of identical hardware, though they leave open how much a customer could save on a particular job through the marketplace.

For a business deciding where to route its next request, a lower price offers little advantage if the cheapest provider leaves employees waiting for a response. “If one provider is underperforming during a particular period, you can simply switch to another. As long as the available providers can deliver inference tokens within an acceptable price range and meet the required latency thresholds, the service can continue without disruption,” Fan says. “Ultimately, it comes down to how you design the routing system and the underlying network to ensure customers receive consistent, reliable service across a group of providers.”

Who Verifies When an AI Agent Spends Money?

Mastercard launched Agent Pay for Machines in June, and Amazon Web Services made agent payments generally available in August. The payment record reveals surprisingly little about who made the decision to spend. A script following a standing instruction can leave the same blockchain entry as an AI agent choosing a service on its own. TRM Labs estimated that just 0.6% to 7.5% of the $25.62 million in payments it examined through the x402 protocol appeared to come from agents.

“I would say the biggest concern for consumers is that they still do not trust AI agents. For example, I cannot simply instruct an agent to manage my account, access the stablecoins or other cryptocurrencies it holds, and begin investing or executing quantitative trading strategies on my behalf,” Fan says. “We still lack sufficient confidence in what these AI agents are capable of and which models they are actually using. That concern exists even with agents designed for legitimate purposes. There is also the risk of malicious agents that could simply transfer tokens out of a user’s account. If we want customers to develop greater confidence in the agents they use, we need to establish ways to verify both the underlying models and the agents themselves.”

An employer looking into a disputed payment would have two separate questions. Did the provider run the model the company approved? Did anyone authorize that model’s proposed purchase? A proof could help answer the first. The employer would still have to account for the authority it gave the agent. The money at risk also changes depending on who asks. A provider might lose its stake over a single inference job while its customer could lose whatever sum the agent had permission to move. Fan didn’t say how the stake would scale with the transaction.

For 2027, Fan wants to bring larger open-source models from families such as Kimi and MiniMax onto CyAI and recruit enough providers to give buyers a choice of hosts for each. The harder engineering target is to extend how much work his team can verify at a price a customer would pay.

“Of course, at this early stage, we are still relying on sampling rather than verifying every inference result. The token window we currently support is around 30,000 tokens, which I believe is sufficient for typical coding tasks and a range of other applications,” he says. “However, we still need to make the technology considerably more practical before zero-knowledge proofs can be deployed on a much broader scale.”

Author:

Victor Dey is a tech analyst and writer who covers AI, data science, startups, and cybersecurity. A former AI editor at VentureBeat, his work also appears in New York Observer, Fast Company, Entrepreneur Magazine, HackerNoon, and more. Victor has mentored student founders at accelerator programs at leading universities including the University of Oxford and the University of Southern California, and holds a Master's degree in data science and analytics.

Follow Victor Dey @iamvictordey