AI & Technology

AI won’t kill us, but we still need to build its seatbelt

By Asmo Urpilainen, CTO, Frends 

Every few months there’s a new wave of headlines declaring that AI is on a collision course with humanity. Recently it was social media’s turn: AI is close to ending human life. The statement was doing the rounds with the confidence of a weather warning. I don’t think that’s really where this goes, but the panic is pointing at something, even if a little clumsily. 

We’ve seen this before  

Cars are the obvious comparison. In the decades after mass adoption, they killed people by the tens of thousands a year, a toll that, given how many cars were on the road, makes today’s driving look almost quaint by comparison. But nobody seriously proposed banning cars. What happened instead was people found solutions: seatbelts, crumple zones, airbags, licensing, crash testing regimes. 

You can trace roughly the same arc through electricity, aviation, pharmaceuticals, the internet. The capability shows up first. Some harm follows, at real human cost. Then, come the guardrails, usually only once they became impossible to avoid politically. So, AI probably won’t kill us. But what we can do this time is act early. 

What’s uncomfortable is that we already know how similar stories end when the guardrails aren’t there from the start.  But that shouldn’t be a reason to shrug and assume it’ll sort itself out either.  

We’ve also done it the other way round  

There’s a version of this that’s comforting mostly because it’s familiar: every technology goes through growing pains, so will this one. That’s fine as far as it goes, but it assumes we have as much time as cars did to work things out. The scale and pace of change has never been this big so this time perhaps we don’t have that luxury. Just like we didn’t when another threat put us all under real time pressure. 

Remember Y2K? Many reading this might have no idea what I’m talking about. At the end of 1999, the world was terrified that every machine would simply stop working when the year’s two digits went from “99” to “00”. The fear was genuinely credible at the time: planes, power grids and banks would all fail at once. It didn’t. But it’s basically what we’re doing now with AI, when a perceived flaw is sitting latent across millions of systems, ready to trigger everywhere simultaneously.  

AI doesn’t have a fixed midnight the way Y2K did, which is part of what makes it harder. But they share underlying risk in a similar shape and form: a flaw in one place can be present everywhere, all at once. A single model update rolls out across millions of workflows simultaneously; customer service, financial decisioning, healthcare triage, infrastructure code and whatever changes, changes everywhere at the same moment.  It ships live, everywhere, immediately. Unlike the cars of yore. 

Any governance framework written for what a model could do at one point in time is usually out of date by the time the next version arrives with capabilities nobody accounted for. AI is the harder version of the same test, because the deadline is basically always now. 

The order of operations matters  

When people talk about “AI safety” the instinct is to jump straight to a values question; what should the system be allowed to do, should it refuse certain requests, should it be honest or careful or deferential. Worthwhile questions, and plenty of good work is going into them. 

But there’s one question that comes before those: what can this thing reach? By that I mean what data it can read, what actions it can take, which systems it can call into and what happens if any of that goes wrong, by accident, by malice or just by something nobody thought to check for. 

Governing behaviour after the fact is a values exercise. Governing access before deployment is an engineering exercise, and it’s one we already know how to do. You don’t need to have solved machine alignment to decide a customer-facing agent shouldn’t have write access to your production database, or that a tool built to summarise internal documents shouldn’t also be able to send email on someone’s behalf, or that an autonomous workflow should hit a human before it does anything irreversible. 

It’s the same logic that eventually made cars survivable: you don’t just tell people to be careful, you build the system, so a moment of carelessness doesn’t turn into a catastrophe. A seatbelt limits what a bad moment can become, it has nothing to do with how carefully anyone drives. AI needs the same kind of limit, and it needs to be there from the start, not added after we’ve exposed our systems to risk. 

The accountability we should aim for 

None of this is an argument for slowing down out of fear, and it’s not an argument that AI is barrelling towards some inevitable disaster either. It’s an argument that whoever builds and deploys these systems has a fairly specific job to do first: work out what the system can reach before deciding what it’s allowed to do. 

Practically, that means treating access as something you design deliberately rather than something that just happens to be broad because broad was easier. It means checking, before anything goes live, exactly which systems and data and actions a given AI agent can touch, and defaulting to the narrowest version that still does the job. It means putting a human in the loop for anything that can’t be undone, before the system is ever trusted to run on its own. And it means being honest that because a single deployment can sit across millions of workflows at once, getting this wrong doesn’t cost you one bad decision, it costs you every decision that model was touching. 

A seatbelt didn’t make cars safe in the abstract. It changed what happened the moment something could have gone wrong. It did that before the crash. That’s roughly the bar AI governance needs to clear. Given the pace things are moving, we’re not going to get the decades of trial-and-error cars took to figure it out. 

The headlines asking whether AI will end human life are asking the wrong question, which is exactly why they get so much traction. I find another question a bit more useful:  what can your AI reach right now?  It’s the one gap we can close today, with tools that already exist. 

 

Related Articles

Back to top button