Cybercrime used to be as predictable as a Bond villain. A clumsy email. A suspect link. A dodgy attachment – all obvious signs of a phishing attempt.
Then AI turned up and suddenly, not only are the typos gone and grammar fixed, the bad actors can use AI to write phishing attacks in the right tone, with your internal phrases, on your project schedule, and that adapt if you reply. And they can deploy them at scale.
Unfortunately, annual security awareness training was designed for a world which doesn’t exist anymore. It’s like taking a first aid course once a year and then being asked to perform surgery on a motorway pile up.
Attacks are relentless and your defence needs to adapt accordingly. In this AI-driven environment, you need live coaching, in the flow of work, that nudges people and systems just before they do something regrettable. If the threat is continuous, your protection needs to be continuous too.
The new workforce
The other problem is that your workforce is no longer just the people in your directory. It is those people and their AI agents, stitched together by SaaS, plugins, API keys, and blind optimism. Those AI agents are already reading inboxes, posting to customers, scheduling money movement, drafting code, and browsing the web on your behalf. And they are very helpful. That is until a criminal convinces an agent to overshare, overreach, or overwrite something that should never have been touched in the first place.
Understandably, this is the point where many organisations try to lock everything down but you can end up in a bit of a stand-off. Security insisting on gates and guards while staff avoid eye contact and use unauthorised or ‘shadow’ tools because they need to get the job done.
That blurry gap between what is allowed and what is necessary is where things go wrong.
Scale and speed
So what changes when AI takes charge and what do we actually do about it?
First, precision at scale. An attacker can now produce a thousand tailored messages that look and feel like your normal day, then iterate based on who bites.
The old red flags hardly ever make it through filters. What passes through looks like normal boring routine work emails. That is why training people to spot bad grammar and suspicious attachments isn’t enough. Phishes will arrive inside an ongoing thread, with the right tone, asking for a small, plausible exception. This is where context becomes far more important than the actual words.
Secondly the speed has changed. These attacks are conversational now. One mail tests the water, the next mirrors your reply, the third closes the loop. And this goes across channels, hopping from email to chat to SMS to a quick Friday afternoon video call.
Controls ahead
When the risk is rising, we need guidance to appear at the right place, at the right time. A just-in-time prompt that says “This attachment is from a new sender and has signs of impersonation – it should be sent to the security team for review”. “A payment detail has changed, confirm over a channel we know you control”. “This thread looks like it has been hijacked, hold it for a quick review”. This keeps the business flowing, but you place guardrails at the moment of choice.
If an agent can touch data, post externally, or trigger spend, treat it like a colleague with very fast hands. Give it an identity. Issue unique credentials that rotate and can be revoked. Keep its permissions minimal and explicit. If it wants to do something risky, make it ask nicely and get a human to sign off. Use sandboxes and verify what the agent thinks happened, actually did.
Log which tools were called, what data was touched, where outputs were sent, and why the agent believed that was reasonable. You will thank yourself when you need to reconstruct how a calendar bot ended up approving a bank change for a vendor you stopped using in 2022.
A positive culture
Most of all, build a positive and strong culture. If your response to mistakes is punishment, your outcome will be silence. People do not report early if they fear the hammer. Reward reporting, praise near misses, and make the secure action the easy one.
Explain the why, not just the rule.
If you do this well, your examples change. Thread hijacking gets spotted because the language and timing don’t fit the sender’s normal cadence. The system pauses the message and offers a one tap verify rather than dumping an attachment into someone’s lap. A vendor asks to update banking details and your workflow requires a callback to a number on file. The agent that filed the request cannot approve it, and the hold quietly expires if verification never lands. A coding agent asks for elevated access, a quick test plan. If approved, the permission evaporates after the job is done. No fear, just sensible friction at the right moment.
Leaning into the workflow
AI has made offence faster, cheaper and, most worryingly, personal. The answer is not to go full luddite or cling to the annual awareness month. Coach people in the flow. Govern agents like colleagues with superpowers. Establish processes that catch the one step you cannot take back.
The workforce is hybrid now, human judgement plus machine capability on the same team. Secure the team as one.
—
Javvad Malik is Lead CISO Advisor at KnowBe4 and still hasn’t found the person who keeps unplugging his YubiKey to charge their phone, but his threat model has a prime suspect.
