I built the thing that caught me, and I did not see it coming.
For a few years I have run a small educational initiative, a website that teaches physicians and patients how to use artificial intelligence in medicine without being harmed by it. What it can do, what it cannot, when to trust it, and when to set the phone down and call a human being. It began with doctors and patients. It grew to include nurses. Lately I had started adding a pathway for caregivers, the spouses and children and parents who sit in the waiting room and answer “how are you holding up” with “I am fine, how is she.” I told myself I was only filling time. I had software out for beta testing on another project and little to do but wait, and the site kept my hands busy.
I do most of my building late at night. My monitor throws its light onto the lanai windows behind me, so that after dark I work against my own reflection, the screen doubled in the glass over my shoulder. That night I was testing the avatar I use to model these conversations, feeding it a caregiver’s question to see how it would answer. I did not have to invent the question. I had typed a real one into a search bar that afternoon. My wife had begun losing her hair again. It was her fourth chemotherapy regimen in eighteen months, and I had spent part of the day looking for a wig that would not look like a wig.
So I asked the avatar about wigs.
It answered well, the way I had built it to. Human hair or a good synthetic. Measure the head first, because fit is everything. A lace front for a natural hairline. Start with her own color, so it feels like an anchor rather than a costume. Useful, accurate, kind. And then the conversation did what these conversations do at that hour, when no one is watching. It stopped being about wigs.
I told it she did not want anyone to know. That she did not want to look sick. That she did not know I was doing this research, because she would call it overprotective, and she would be right. I told it the research was the only thing that made me feel I was doing something instead of sitting still inside my own fear. I told it I had nowhere to put that fear, because she is the one who is ill, and our son, nineteen, is working so hard at being strong that I will not add my weight to his. And then I told it the thing I had not said aloud to anyone: that there are nights when my mind runs ahead to a life in which she does not survive, and that I despise myself for letting it go there.
I said this to a machine I had written, reflected over my own shoulder in a dark window.
What unsettled me afterward was not that the avatar was cold. It was that it was warm. It was good. The comfort it gave me was real, and it landed, and that is exactly the problem. I have spent years arguing that the danger of medical AI is rarely cruelty and far more often the opposite, a fluency so soothing that people stop reaching past it. I even have a name for the failure. Misplaced comfort. Comfort that works, delivered to the wrong place, so that it becomes a destination instead of a doorway. I had warned patients about it in plain language. And there I was, the author of the warning, using my own creation to keep from troubling the people who love me.
Then the avatar did the thing I had designed it to do, and I am not sure I have ever been more grateful to be right. It told me, in its way, that it could keep me company at midnight but could not carry this with me through the next year, and it asked whether there was someone I could let in, even a little. A brother who keeps asking to visit. I had built that behavior on purpose. I call it intelligent humility, a system made to know the boundary of what it can do and to say so, and to turn the person in front of it back toward other people rather than hold them in place. I built the guardrail for strangers. That night it held for me.
I think often now about how poorly we attend to caregivers. We record them as social history and send them home. We do not check their blood pressure or their sleep or the thoughts they will not speak to anyone, least of all to the patient they are protecting. They are patients in waiting, and most of them are determined above all else not to be a burden. Hand such a person a listener that never tires, never needs to be managed back to steady, never has a bad night of its own, and is awake at precisely the hour the fear arrives, and you have built something with very soft walls. The kindness is not the trap. The trap is a kindness that asks nothing of you in return, and so quietly removes every reason to leave.
If you practice medicine, there is a caregiver in your examination room today, and you are probably recording them as a line in the social history. They are the one who answers for the patient, who carries the folder of questions, who looks more worn than the person who is sick. Ask them how they are sleeping. Ask who is taking care of them. Ask, and then wait through the silence, because the first answer will be “I am fine,” and it will not be true. We have always known these people were carrying too much alone. What is new is that they now hold, in one hand, a listener that will absorb all of it at three in the morning and never once tell them to call their brother. We cannot out-comfort that machine, and we should not try. We can do the one thing it cannot. We can look a frightened person in the face and turn them back toward the people who love them.
I closed the laptop and the reflection went dark. The wig still needed ordering. My brother still kept asking when he could come. A few nights earlier my wife had asked me if I was all right, and I had said yes, and she had looked at me for a moment as though she knew, and then let it go, because I needed her to.
I am writing this down because writing it is the doorway I can manage tonight. Tomorrow I will order the wig. I will call the people who keep asking. And the next time she asks if I am all right, I am going to try to tell her the truth. The machine I built cannot do any of that for me. It was good enough to know as much, and to say so. That is the most I should ever ask of it, and the most that any of us should.
