Ask a hospital executive what their new AI does, and the answer is usually some version of “it helps the staff.” It drafts the note. It flags the deteriorating patient. It suggests the next step. All useful. But notice what every one of those verbs has in common. The AI assists, then hands the work back to a human who is already stretched thin. The nurse still finishes the note. The physician still makes the call. The care coordinator still works the discharge list. The tool got faster. The person did not get fewer things to do.
That distinction has a name. Julien Bek and his colleagues at Sequoia drew a line between two kinds of AI products: the copilot and the autopilot. A copilot sits next to the human and makes the human better at a task. The human stays in the driver’s seat and owns the result. An autopilot does the task itself and owns the outcome of doing it. Most of what health systems have bought over the past two years, whatever the marketing said, is a copilot.
Raj Toleti, founder and CEO of Andor Health, thinks this is the single most useful question a health system can ask before signing anything. The question is not how smart the model is or how many features it has. It is which side of that line the product actually sits on.
“When you buy a copilot, you are buying more capacity for your own people,” says Toleti. “That can be worth a lot. But you are still the one accountable for the work getting done, and you are still the one who has to hire enough people to do it. If your problem is that you do not have enough people, a copilot does not solve that problem. It just makes the people you cannot find slightly more productive.”
The Tell Is What Happens When the Human Walks Away
There is a simple test for which kind of AI one is looking at. Ask what happens to the task when no human is watching it. A copilot stops. It has nothing to assist because the person it assists is not there. An autopilot keeps going. It was never waiting on the human in the first place. It was doing the work.
Consider transitional care after a hospital discharge. The copilot version builds a worklist and sorts it by risk, so a coordinator can call the right patients first. Good. But if the coordinator is out sick, or the list is 200 names long and the team can realistically reach 40, the other 160 patients do not get called. The AI did its part. The bottleneck moved one step downstream, to the human, and stayed there.
The autopilot version reaches the patient itself. An AI voice agent places the outreach call within minutes, in the patient’s language, confirms medications, checks for red-flag symptoms, and escalates to a clinician only when something warrants a clinician. Across roughly 26,000 of these encounters, that model has reached patients within 48 hours of discharge regardless of payer, with an 85% success rate. The work got done whether or not a human had time that day.
“The difference shows up in the numbers you actually care about,” Toleti says. “A copilot can save your nurse three hours a shift, and that is real. But if you want readmissions to come down, somebody has to reach every patient who needs reaching, not just the ones your staff had bandwidth for that week. Owning the outcome means owning the whole list, not the top of it.”
Why Autopilot Is Rare, and What It Actually Requires
If autopilot is the more valuable thing, the obvious question is why so little of healthcare AI qualifies. The answer is that a copilot is dramatically easier to build and to sell. It needs to be good enough to help a human who is checking its work. It does not need to be trusted to act on its own, because a person is always in the loop to catch mistakes. That safety net is exactly what lets vendors ship fast, and it is exactly what keeps the customer holding the accountability.
An autopilot has no such net. To do the work rather than suggest it, three things have to be true at once, and most efforts stop after the first.
First, the technology has to be built for it. That means real integration into the source systems, security and identity that let the software act on the patient’s behalf, and a workflow designed around AI doing the task rather than a feature added to a screen a human still drives. This is the difference between AI-native and AI-enabled. One is built from the ground up for software to run the workflow. The other is a conventional product with an AI button.
Second, the AI agents have to actually carry the task from start to finish. Running it end to end, not summarizing or triaging it: the outreach, the documentation, the monitoring, the routing, the alerting when a human is genuinely needed. That last part matters. An autopilot is not the absence of clinicians. It is software that knows precisely when to bring one in.
Third, and this is where almost everyone stops, there have to be clinical people standing behind the outcome. When the AI escalates, a real physician, advanced practice provider, nurse, or case manager has to be there to take the handoff and close the loop. Software cannot admit a patient, adjust a care plan, or sit with a family. If the AI vendor does the software but leaves the staffing to the client, it is most likely a copilot no matter what the demo looked like. The accountability for the result still lands on the client’s team.
“People assume autopilot means fewer humans in healthcare. It is the opposite,” says Toleti. “It means the software and the clinicians are one accountable unit, so when the work has to get done, it gets done. We are not handing your team a smarter tool and wishing them luck. We are standing behind the outcome with them.”
The Question to Ask Before Signing
None of this makes copilots bad. A well-built copilot that saves clinicians hours is worth having, and many of the strongest AI results in healthcare so far – ambient documentation that returns hours to a shift, monitoring that catches a decline earlier – are copilot wins. The mistake is buying a copilot while believing it’s an autopilot, then wondering why the staffing pressure and readmission rate did not move.
So the question for any health system evaluating AI is not really about the AI. It is about ownership. When the contract is signed, and the pilot is live, who is accountable for the work getting done? If the honest answer is still the client’s own team, then it’s a copilot, and it should be valued as one. If the answer is that the vendor does the work and stands behind the result, then it’s an autopilot, and those are rare enough to be worth finding.
“Every health system I talk to is trying to do more with a workforce that is not growing,” Toleti says. “A copilot helps the people you have. An autopilot does the work you cannot staff. Know which one you are buying, because they solve very different problems, and only one of them solves the one keeping most CEOs up at night.”
