AI & Technology

Where do we go now: Rogue AI models, rapid advancements and identity verification

By Ricardo Amper, Founder & CEO of Incode Technologies

The headlines around Anthropic’s advanced Mythos AI model creating fake profiles to deceive people and attempting to use ‘malicious code’, or OpenAI’s Hugging Face Hack didn’t necessarily come as a surprise to those who have been following these models and their developments. 

Whereas for the general public, these advancements were likely alarming and dystopian. Which we can see reflected in how these stories were reported on and the comments section.  

For both parties, these incidents came with a lot of questions about where we go from here. We can’t just brush these under the rug simply because we’d rather celebrate our technological advancement. Nor should we throw away all the hard work and innovation that’s been put into these models just because we’ve come up against a real challenge. 

Surprisingly the answer to this question has less to do with the agents, and more about us.  

So how did we get here?

There is a reason the most advanced form of AI we have right now is language models. Code is also a language. It has structure, patterns, logic, repetition, exceptions, all of which these models understand, for better or worse. It is also why they are becoming so useful in software development, debugging, and now vulnerability discovery. AI can look at massive amounts of code and spot patterns or inconsistencies much faster than a person can.  

The thing about the Mythos incident was that the agent forged nothing. It didn’t pick the identity lock, because there was no lock to begin with, anyone can register on a forum or similar platform by just filling out a form. The cyber incidents of August 2026 should be a stark reminder that our old models of: verify once, trust for a while no longer work. 

As AI gets better at mimicking people, passing simple checks, and operating at scale, the question is no longer just who got in at the start. The question is whether the thing interacting with the system right now is still a real human. That is why continuous human verification matters more now. If we keep introducing friction that AI cannot easily pass – especially around biometric presence, liveness, and proof of a real person behind an action – we make it much harder for synthetic actors to operate freely at scale. 

Organisations need to move to verifying against sources fraudsters can’t fabricate, rather than sit around waiting for the next headline. 

Advancements, both good and bad 

Advancements in AI and language models cut both ways. It helps defenders move faster, but it also means attackers will have much more powerful tools for reconnaissance, discovery, and adapting their methods quickly. 

It is one thing to build something powerful. It is another thing to be able to contain, control, and make sure the right people have the right level of access at every point around it. This is where the industry is still catching up. Labs are moving incredibly fast, and the culture of speed, experimentation, and constant iteration does not naturally come with the same discipline you need for securing something this sensitive. 

If AI is going to act on behalf of people, access systems, move through workflows, or make decisions that affect real customers, there has to be infrastructure that can confirm who is behind that action, whether a real human is present when needed, and whether the behavior matches a legitimate identity context. 

That infrastructure has to be continuous, not one-time. It needs strong biometric verification, liveness, session integrity, device and behavior signals, and clear ways to step up trust when the risk gets higher. Otherwise, we are building very powerful systems on top of very weak assumptions about who or what is actually interacting with them. 

Up next… 

The stakes are high right now, because this is not just about one model of one company. What we’re seeing is that advanced AI is starting to affect both sides of the security equations at the same time. What that means is that security and identity can no longer be treated as a separate conversation. The more powerful AI becomes, the more important it is to know who is real, who is authorised, and when a machine is acting without a human behind it. 

While much of the conversation has been focused around ‘AI behaving badly’ making the kneejerk reaction to call for it to be ‘stopped’ or ‘shut down’ the real takeaway here is that these cyber incidents are only proving to us that there was no meaningful identity check within the system. There was no lock, let alone an in-person bouncer or a high-tech security system.  

Our next step shouldn’t be to ban agents, in a similar way to how we didn’t ban cars because they were being stolen. Instead we found solutions – introduced plates, licenses, insurance. Agents need the same accountability infrastructure that verifies identity, clearly defines authority, and has an accountable person or organisation behind every single consequential action.  

Human beings are resilient, we know how to find solutions. Now is the time to act on them. 

Related Articles

Back to top button