AI

Sora 2: Deepfakes Waiting to Happen

By John Wilson, Senior Fellow, Threat Research at Fortra

Sora 2,ย OpenAIโ€™sย advanced model for generating realistic, high-quality videos from text or images, is being positioned as a breakthrough in video generation. OpenAI touts it asย โ€œmore physically accurate, realistic, and more controllable than prior systems.โ€ It is capable of synchronized dialogue, sound effects, and inserting real people into generated scenes with remarkable fidelity.ย ย 

Whileย OpenAIย frames it as a new era of creativity and communication, when viewed through a security lens, Sora 2 introduces a deeply uncomfortable problem: the easier it becomes to create realistic videos of real people, the easier it becomes to make them appear to do and say things they never did.ย ย 

By default, the cameos created in Sora 2 are available for anyone to use. If you do not change this setting, anyone could create a realistic video of you doing and saying things that would be out ofย characterย or which others might find offensive.ย 

That single design choice dramatically shifts the threat landscape.ย What onceย requiredย sophisticated deepfake tooling and technicalย expertiseย is now packaged into a consumer-friendly app, complete with a social feed designed for remixing and reuse.ย 

Likeness,ย Metadata, and the Longย Tail of Exposureย 

When you upload photos and videos to Sora, the default is toย retainย any location information embedded in the media. Most social platforms strip that data out. However, even if the location information is stripped from the final created content,ย thereโ€™sย always a risk of a data breach leading to information disclosure at some point in the future.ย ย 

Identity, likeness, voice, and metadata are all being pooled into a system designed for scale.ย 

OpenAI highlights that you canย โ€œdrop yourself straight into any Sora sceneโ€ after a one-time recording to capture appearance and voice. The capability is very general, as it works for any human, animal, or object.ย ย 

From a creative standpoint, that is impressive. From a social engineering standpoint, it is incendiary.ย 

Romanceย Scams Get aย Powerful New Toolย 

Romance scamsย are an obviousย beneficiary. Sora 2 and similar technologies will prove to be a highly effective tool for romance scammers, who currently go to great lengths to convince their victims that they are wealthy and always on the go.ย ย ย 

With Sora 2, a scammer can send videos of themselves sailing on a mega yacht, driving an exotic car, or flying in a private jet with just a few keystrokes.ย 

Once they develop a cameo of their persona, they can make that cameo do or sayย just about anything. Video, long treated as higher-trust evidence than text or images, becomes a persuasion engine that is far harder for victims to question.ย 

Business Scams are Not Here Yet, But They Will Beย 

At first glance, theย impact on traditional businessย scamsย appears limited. Weย don’tย expect Sora 2 to have much of an impact on businessย scamsย such as payment diversion, since most payment discussions happen over email or voice, rather than via video.ย ย 

Also, Sora 2 videos alsoย arenโ€™tย live; they must be pre-generated, which limits their usefulness for real-time impersonation. Yet that limitation is temporary. At the rate the technology isย advancing,ย however, we imagine Sora being able to generate near instantaneous video from text prompts in theย relatively nearย future.ย 

When that happens, the threat model shifts overnight. We expect to see all manner of abuse, such as yourย โ€˜bossโ€™ย calling you on Teams to ask you to wireย a large sumย of money to an offshore account.ย 

Employees as the New Attack Surfaceย ย 

Even before real-time video becomes a reality, the business risks go beyond finance. Employees are users who have public profiles, recorded talks, internal presentations, and now,ย possibly Soraย cameos. If those cameos are public, they can be reused (and abused) in ways those users never intended.ย 

Enterprises should discourage their staff from making their Sora 2 cameos public because they could be used to spread disinformation about the organization. The examples are frightening for a reason. Imagine the danger of a well-known CFO appearing to say that sales are down 50% from the prior quarter, or the impact of your CEOย appearing to beย drunk and verbally abusive in public.ย 

Sora 2 is explicitly touted as being good at realism, dialogue, and persistence of world state. OpenAI claims it can followย โ€œintricate instructions spanning multiple shots while accurately persisting world state.โ€ย ย ย 

That makes fabricated footage more believable, more shareable, and farย moreย damaging.ย ย 

Marketing Temptation and the Erosion of Trustย ย 

There is also a quieter internal risk. It might be tempting for your marketing team to generate video content using your subject-matter experts’ likenesses. At scale, that sounds efficient, but withoutย controls, it becomes dangerous.ย 

For entities that choose this route, the warning is clear: require the subject of the video to sign off on any content before it is published. Otherwise, your expert might be saying things that simplyย aren’tย true on your website, effectively eroding public trust in them.ย 

Guardrails that Areย Easy to Step Aroundย ย 

Sora 2 claims to have controls around creating characters using life-like human images. In practice, this is easily bypassed by first converting the image to a cartoon or other artistic style using another AI tool, then using the stylized image to create the character.ย ย 

The assumption that a stylized image is harmless breaks down once motion, voice, and context reintroduce realism.ย 

Experiments with public figures show inconsistent enforcement. Sora 2 generated realistic likenesses of certain deceased presidents without issue, blocked others, and applied different policy rationales to different figures.ย The result is a system that is difficult to predict and easy to probe.ย 

Consent isย Not the Same asย Controlย 

OpenAI emphasizes thatย โ€œwith characters, you are in control of your likeness end-to-end with Sora.โ€ You can revoke access and remove videos. However, this assumes that users understand the defaults andย the risks.ย ย 

Once a convincing video has circulated, the damage to reputation may be done, regardless of whether the original asset is later removed.ย 

The issue with Sora 2 is not that it enables creativity, but that it lowers the bar for misuse. When realistic video can be generated from a cameo, and that cameo can be reused by others, control over how a person appears in public is no longer guaranteed.ย 

Even if it does notย immediatelyย change how traditional businessย scamsย operate, it introduces a new andย largely unaddressedย risk: employees and leaders can be made to appear to say or do things they never did, with real consequences for trust, reputation, and credibility.ย 

Asย the technologyย moves closer to real-time generation, the opportunity for abuse only increases. At that point, video is no longer something organizations can treat as inherently trustworthy; it becomes another channel that must be actively defended.ย 

Author

Related Articles

Back to top button