Deepfake experts warn AI is making fake content harder to detect

“Nowadays I have a hard time telling what is real from what is synthetic.”

Sam Gregory is an expert in deepfakes – audio or visual content generated or manipulated using AI that usually misrepresents someone or something.

Yet advances in generative AI mean that even he is regularly fooled by them. He helped build WITNESS’s Deepfake Rapid Response Force, which examines suspected footage on behalf of journalists and investigators. He says he’s fighting a losing battle.

The tools designed to detect synthetic content are struggling to keep pace. Gregory argues many are unreliable and inaccurate – leaving democracies uniquely exposed to misinformation campaigns.

“For many years I told people to prepare, don’t panic,” he says. “Now I wonder whether it’s time to panic.”

There’s nothing new about these kinds of warnings.

Previous US elections have been accompanied by grim predictions of political deepfakes. Before the presidential elections in 2020 there were a string of articles alerting voters to inauthentic videos of politicians – fakes that could undermine the democratic process.

Yet the deepfakes in that election never quite lived up to the fear. NPR even wrote a piece wondering where the deepfakes were.

The 2024 US presidential election was also touted as the first “deepfake election”. Their impact barely registered as an issue.

It was left to NPR again to correct the record. “The feared wave of deceptive, targeted deepfakes didn’t really materialize,” a reporter concluded.

Malicious deepfakes just weren’t quite good enough – yet. Until recently, we have generally been able to tell what was real and what was fake. “For a long time it felt like deepfakes were overhyped,” Gregory says. Then about two years ago, with rapid technological advances in AI, that began to change.

In 2024, the Bureau followed a scam network running altered videos of former British Prime Ministers Keir Starmer and Rishi Sunak on Facebook and Instagram. Whoever made them had edited both men so that they appeared to be discussing the details of a high-return con investment scheme. People in the UK alone watched them millions of times.

Since then, AI-generated video and audio have become cheaper, faster and more convincing. During our work on AI slop we’ve watched AI models churn out more and more realistic video and audio. We now regularly struggle to tell whether a video has been created with AI or not.

But what’s surprising is that even experts like Sam Gregory also get it wrong.

“I got fooled by The Pope in a puffer jacket,” he says. “I saw him, and I thought that’s a cool image of the Pope.”

Yet advances in the ability to create deepfakes have not been matched by developments in detection technology. Tools tend to indicate the likelihood that a visual is synthetic, but can rarely say with 100% certainty.

“The thing to know about detection tools is that they are often unreliable, particularly on content that is compressed, or in languages that a tool for detection wasn’t trained on.”

Gregory gives a recent example from Iran: a viral video showing images of a protester facing up security forces. It was real.

“Then someone chose to enhance that image with AI, to make it clearer, to make it more shareable, to maybe understand the details more. And as a result, what happened was it then triggered AI detection tools”.

Examples like this can actually make deepfake detection harmful – actively enabling real images to be painted as fakes and allowing governments to weaponise misinformation.

It gives politicians like Donald Trump a dream scenario: If you want to deny a real video, just claim it’s AI.

We know the US President thinks this, because he literally said so.

Last year, a video of someone throwing a rubbish bag out of a White House window went viral. Asked about it in the Oval Office the next day, Trump told reporters it wasn’t real and was “probably” made with AI. The White House had already told reporters the truth, that the footage was genuine and showed a contractor doing maintenance while the president was away.

Then, seemingly as if the idea had just come to him, he said: “If something happens that’s really bad, maybe I’ll have to just blame AI,” he said.

The problem is, this strategy often works. It takes far longer to prove something is real than it is fake – by which time the falsehood has already done the rounds on social media.

And advances in deepfakes aren’t just helping politicians who want to obfuscate the truth.

Lawyers have made much the same argument in court, telling judges that video evidence and recordings (which had already been checked and confirmed) might be fakes after all.

Earlier this year Grok showed how AI generation tools could be weaponised against women en masse. It took weeks for xAI and Elon Musk to respond, even then falling short of stopping the chatbot creating sexualised deepfakes.

Gregory argues that social media companies have become worryingly complacent about deepfakes. “They seem to care very little about the collateral damage. It’s all part of a trajectory towards a bigger vision.”

As a result, people trust what they see and hear online less – as the media they consume is flooded with fakes. On this point, in our latest film, Gregory quotes the philosopher Hannah Arendt:

“A people that can no longer believe anything cannot make up its mind. It is deprived not only of its capacity to act but also of its capacity to think and to judge. And with such people you can then do what you please.”